a_adli/colouring-montreal
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

remove use of addressbase

Browse Source
This commit is contained in:
Ed Chalstrey 2022-03-11 11:46:00 +00:00
parent 7f3693355e
commit 6916e4ca59
2 changed files with 9 additions and 9 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
etl/README.md
View File

@ -59,11 +59,11 @@ The scripts should be run in the following order:
cd ~/colouring-london/etl cd ~/colouring-london/etl
``` ```
Extract the addressBase dataset. <!-- Extract the addressBase dataset.
```bash ```bash
sudo ./extract_addressbase.sh ./addressbase_dir sudo ./extract_addressbase.sh ./addressbase_dir
``` ``` -->
<!-- ERROR 1: Couldn't fetch requested layer 'BasicLandPropertyUnit'! --> <!-- ERROR 1: Couldn't fetch requested layer 'BasicLandPropertyUnit'! -->
@ -84,7 +84,7 @@ source colouringlondon/bin/activate
Filter MasterMap 'building' polygons and any others referenced by addressbase. Filter MasterMap 'building' polygons and any others referenced by addressbase.
```bash ```bash
sudo ./filter_transform_mastermap_for_loading.sh ./addressbase_dir ./mastermap_dir sudo ./filter_transform_mastermap_for_loading.sh ./mastermap_dir
``` ```
Load all building outlines. Load all building outlines.

12
etl/filter_transform_mastermap_for_loading.sh
View File

@ -4,27 +4,27 @@
# Filter and transform for loading # Filter and transform for loading
# #
: ${1?"Usage: $0 ./path/to/addressbase/dir ./path/to/mastermap/dir"} : ${1?"Usage: $0 ./path/to/addressbase/dir ./path/to/mastermap/dir"}
: ${2?"Usage: $0 ./path/to/addressbase/dir ./path/to/mastermap/dir"} # : ${2?"Usage: $0 ./path/to/addressbase/dir ./path/to/mastermap/dir"}
addressbase_dir=$1 # addressbase_dir=$1
mastermap_dir=$2 mastermap_dir=$1
# #
# Check which TOIDs are matched against UPRNs # Check which TOIDs are matched against UPRNs
# #
colouringlondon/bin/python check_ab_mm_match.py $addressbase_dir $mastermap_dir # colouringlondon/bin/python check_ab_mm_match.py $addressbase_dir $mastermap_dir
# #
# Filter # Filter
# - WHERE descriptiveGroup = '(1:Building)' # - WHERE descriptiveGroup = '(1:Building)'
# - OR toid in addressbase_toids # - OR toid in addressbase_toids
# #
colouringlondon/bin/python filter_mastermap.py $addressbase_dir $mastermap_dir # colouringlondon/bin/python filter_mastermap.py $addressbase_dir $mastermap_dir
# #
# Transform to 3857 (web mercator) # Transform to 3857 (web mercator)
# #
find $mastermap_dir -type f -name '*.filtered.csv' -printf "%f\n" | \ find $mastermap_dir -type f -name '*.gml.csv' -printf "%f\n" | \
parallel \ parallel \
ogr2ogr \ ogr2ogr \
-f CSV $mastermap_dir/{}.3857.csv \ -f CSV $mastermap_dir/{}.3857.csv \