luis rodríguez lado e-mail : [email protected]

1 JRC – Ispra – 23 July 2004

Luis Rodríguez Lado

E-mail : [email protected]

Alpine Soil Information SystemAnalysis of the accuracy of ESBD in the Alps

region


Introduction

There is a increasing demand of soil maps and of their properties in the frame of the EU. This information is needed to develop policies linked to sustainable land management practices, and to avoid the damage risk to ecosystems.

At present, the 1:1M digital soil map and of some of their properties are available at European Scale.


Objective

In this exercise, we evaluate the accuracy of the ESDB maps by comparison with some reference maps derived from detailed survey (ECALP).


Accurate digital soil maps were computed for 5 pilot areas in the Alps region (ECALP Project).

Methodologydata from ECALP areas


Maps in the ECALP areas are available as to raster based soil maps.The pilot areas were divided in 1Km2 cells. In this analysis, the soil properties used for each cell are those of the main Soil Map Unit in the cell (% area).

Methodologydata from ECALP areas


The 1:1M ESDB was rasterized into a 1Km2 cell raster grid.The soil properties for each grid cell were also those of the main Soil Map Unit in the cell (% area).

We compare the results of both maps.

Methodologydata from ESDB areas


Texture.

Depth of presence of an obstacle to roots.

Depth of presence of an impermeable layer.

Methodologyproperties analyzed


Methodology

The accuracy of the 1:1 M map is expressed by “naïve” measures of accuracy using “confusion matrices”.


Objective

PRUEBA ECALP

Cat1 Cat2 Cat3 Cat4 Cat5 Cat6 Cat7 Cat8 Cat9 Cat10Total pi+ User accuracy Producer accuracy Kappa User Kappa Producer

ESDB

Cat14401 37 0 294 13 118 4863 0,9050 0,8008 0,8693 0,7372

0,9050 0,0076 0,0000 0,0605 0,0027 0,0243

Cat234 541 231 1263 51 0 2120 0,2552 0,4055 0,2023 0,3355

0,0160 0,2552 0,1090 0,5958 0,0241 0,0000

Cat30 284 700 542 143 0 1669 0,4194 0,3667 0,3585 0,3094

0,0000 0,1702 0,4194 0,3247 0,0857 0,0000

Cat41054 454 182 4409 130 15 6244 0,7061 0,5938 0,5341 0,4109

0,1688 0,0727 0,0291 0,7061 0,0208 0,0024

Cat57 18 782 851 626 309 2593 0,2414 0,6298 0,2020 0,5750

0,0027 0,0069 0,3016 0,3282 0,2414 0,1192

Cat60 0 14 66 31 2507 2618 0,9576 0,8501 0,9503 0,8277

0,0000 0,0000 0,0053 0,0252 0,0118 0,9576

Cat7 0

Cat8 0

Cat9 0

Cat10

0

Total 5496 1334 1909 7425 994 2949 20107

p+j

Overall Accuracy = 0,6557 Global Kappa = 0,557939

S.D. = 0,0034


Methodology

The User’s accuracy expresses the probability that one class (in ESDB) is well mapped in relation to the reference dataset (ECALP).

The Producer’s accuracy indicates the proportion of cells that were correctly classified.


Objective

PRUEBA ECALP


ESDB

Cat14401 37 0 294 13 118 4863 0,9050 0,8008 0,8693 0,7372

0,9050 0,0076 0,0000 0,0605 0,0027 0,0243

Cat234 541 231 1263 51 0 2120 0,2552 0,4055 0,2023 0,3355

0,0160 0,2552 0,1090 0,5958 0,0241 0,0000

Cat30 284 700 542 143 0 1669 0,4194 0,3667 0,3585 0,3094

0,0000 0,1702 0,4194 0,3247 0,0857 0,0000

Cat41054 454 182 4409 130 15 6244 0,7061 0,5938 0,5341 0,4109

0,1688 0,0727 0,0291 0,7061 0,0208 0,0024

Cat57 18 782 851 626 309 2593 0,2414 0,6298 0,2020 0,5750

0,0027 0,0069 0,3016 0,3282 0,2414 0,1192

Cat60 0 14 66 31 2507 2618 0,9576 0,8501 0,9503 0,8277

0,0000 0,0000 0,0053 0,0252 0,0118 0,9576

Cat7 0

Cat8 0

Cat9 0

Cat10

0

Total 5496 1334 1909 7425 994 2949 20107

p+j


S.D. = 0,0034


Methodology

The Overall accuracy is the sum of the correctly classified cells (diagonal values) divided by the total number of cells analyzed. It indicates the proportion in which those maps agree.

The KAPPA coefficient of agreement is a measure of the chance in the agreement. It indicates whether the agreements found in the overall accuracy are due to the map accuracy of due to chance.


Objective

PRUEBA ECALP


ESDB

Cat14401 37 0 294 13 118 4863 0,9050 0,8008 0,8693 0,7372

0,9050 0,0076 0,0000 0,0605 0,0027 0,0243

Cat234 541 231 1263 51 0 2120 0,2552 0,4055 0,2023 0,3355

0,0160 0,2552 0,1090 0,5958 0,0241 0,0000

Cat30 284 700 542 143 0 1669 0,4194 0,3667 0,3585 0,3094

0,0000 0,1702 0,4194 0,3247 0,0857 0,0000

Cat41054 454 182 4409 130 15 6244 0,7061 0,5938 0,5341 0,4109

0,1688 0,0727 0,0291 0,7061 0,0208 0,0024

Cat57 18 782 851 626 309 2593 0,2414 0,6298 0,2020 0,5750

0,0027 0,0069 0,3016 0,3282 0,2414 0,1192

Cat60 0 14 66 31 2507 2618 0,9576 0,8501 0,9503 0,8277

0,0000 0,0000 0,0053 0,0252 0,0118 0,9576

Cat7 0

Cat8 0

Cat9 0

Cat10

0

Total 5496 1334 1909 7425 994 2949 20107

p+j


S.D. = 0,0034


Methodology

For example:An Overall Accuracy of 0.655 indicate that both maps agree in 65% of the cases.

A Kappa statistic of 0,557 indicates that 55,7% of this agreement is due to the mapper competency, and 9,3% of the agreements were due to chance.


Methodology

Low values of Kappa indicate :

a) Bad Map. Errors due the mapper or to the mapping technique. We can do another map with the same accuracy simply by random assignation using the same classes.

b) An highly homogeneous area (1 class in whole area). For these areas, high values of agreement can be achieved also randomly.


Results


Texture

Resultsfrequency distribution (n = 1818 cells)

0

250

500

750

1000

1250

1500

0 1 2 3 4 5 6 7 8 9

Text ECALP

Text ESDB


Texture

confussion and probabilities matrices; Accuracy index

N

EW

S

Peat SoilsFineMedium-fineMediumCoarseNo Information

Texture ESDB


Texture class Lombardia-Switzerland

N

EW

S


Texture ESDB

N

EW

S

Texture ECALPNo InformationCoarseMediumMedium-fineFinePeat Soils

N

EW

S


(ECALP)(ESDB)


Texture Lombardia


N

EW

S


Texture ESDB


Texture class Friuli-Slovenia

N

EW

S


(ECALP)(ESDB)


Texture Friuli


N

EW

S


Texture ESDB


Conclusions

Texture

0,0000

0,1000

0,2000

0,3000

0,4000

0,5000

0,6000

0,7000

0,8000

0,9000

1,0000

Lombardia Piemonte Friuli Veneto Austria Total

Overall Accuracy

Global Kappa


Conclusions

User's accuracy

0,0000

0,2000

0,4000

0,6000

0,8000

1,0000

1,2000


No information

Coarse

Medium

Texture

Kappa User's accuracy

0,0000

0,0500

0,1000

0,1500

0,2000

0,2500

0,3000

0,3500

0,4000

0,4500


No information

Coarse

Medium


ConclusionsTexture

Kappa Producer's accuracy

0,0000

0,0500

0,1000

0,1500

0,2000

0,2500

0,3000

0,3500

0,4000

0,4500

0,5000


No information

Coarse

Medium

Producer's accuracy

0,0000

0,2000

0,4000

0,6000

0,8000

1,0000

1,2000


No information

Coarse

Medium


Depth of an obstacle for roots


0

250

500

750

1000

0 1 2 3 4 5 6

ROO ECALP

ROO ESDB


Depth of an obstacle for roots

confussion and probabilities matrices; Accuracy indexes

Depth class of an obstacle to roots (ECALP)

Roo ECALPNo informationNo obstacle 0-80 cmObstacle 60-80 cmObstacle 40-60 cmObstacle 20-40 cmObstacle 0-80 cmObstacle 0-20 cm

N

EW

S


Depth to obstacle to roots

Lombardia-Switzerland

(ECALP)(ESDB)



N

EW

S



N

EW

S

Depth class of an obstacle to roots (ESDB)

No informationNo obstacle 0-80 cmObstacle 60-80 cmObstacle 40-60 cmObstacle 20-40 cmObstacle 0-80 cmObstacle 0-20 cm

Roo ESDB

N

EW

S


Conclusions

Obstacle to roots

0,0000

0,1000

0,2000

0,3000

0,4000

0,5000

0,6000

0,7000

0,8000

0,9000


Overall Accuracy

Global Kappa


Conclusions

User's accuracy

0,0000

0,2000

0,4000

0,6000

0,8000

1,0000

1,2000


No information

No obstacle 0-80 cm

Obstacle 60-80 cm

Obstacle 40-60 cm

Obstacle 20-40 cm

Obstacles to roots

Kappa User's accuracy

0,0000

0,2000

0,4000

0,6000

0,8000

1,0000

1,2000


No information

No obstacle 0-80 cm

Obstacle 60-80 cm

Obstacle 40-60 cm

Obstacle 20-40 cm


ConclusionsObstacles to roots

Kappa Producer's accuracy

0,0000

0,1000

0,2000

0,3000

0,4000

0,5000

0,6000

0,7000

0,8000

0,9000


No information

No obstacle 0-80 cm

Obstacle 60-80 cm

Obstacle 40-60 cm

Obstacle 20-40 cm

Producer's accuracy

0,0000

0,1000

0,2000

0,3000

0,4000

0,5000

0,6000

0,7000

0,8000

0,9000

1,0000


No information

No obstacle 0-80 cm

Obstacle 60-80 cm

Obstacle 40-60 cm

Obstacle 20-40 cm


Depth of an impermeable layer


0

500

1000

1500

2000

0 1 2 3 4

IL ECALP

IL ESDB



Results confussion and probabilities matrices; Accuracy index

(n = 1818 cells)


Conclusions


0,0000

0,1000

0,2000

0,3000

0,4000

0,5000

0,6000

0,7000

0,8000

0,9000


Overall Accuracy

Global Kappa


Conclusions

We found that the present 1:1M ESDB maps means a great generalization of soils and their properties, being inappropriate to derive effective policies in the EU at medium and large scales due to the uncertainty of its information.The overall accuracy of these maps is generally lower than 50%. It varies between 0,33 (obstacle to roots) to 0,8 (depth of impermeable layer) but low values of Kappa were found, indicating high influence of chance in the success of classification.This low values of Kappa are greatly due to the low discrimination in classes in ESDB (general map).


Conclusions

Friuli-Slovenia was the region that showed a better agreement with the ECALP database, particularly for the depth of an obstacle to roots, where it also exhibits a high value of Kappa.


Conclusions

1. Need of more accurate soil maps than ESDB

2. Provide soil sample description as metadata

3. Consensus in the description of properties

4. Implementation of accuracy tests for maps

luis rodríguez lado e-mail : [email protected]

Documents