Dominic H
|
9943bdd9a5
|
Test load to staging
- Copy of main load_csv.py with edit to get round ssh error
|
2019-10-02 16:35:56 +01:00 |
|
Tom Russell
|
46ac6c7a40
|
Fix load_csv script - tested against localhost
|
2019-10-02 09:43:12 +01:00 |
|
Tom Russell
|
c055bea38b
|
Add generic CSV upload script (rename load_data to load_shapefile)
|
2019-09-30 10:39:43 +01:00 |
|
Maciej Ziarkowski
|
2c9b5ea3d8
|
Modify routes, refactor API structure
|
2019-08-14 14:05:49 +01:00 |
|
Tom Russell
|
482ab5060c
|
Set postcode zoom and class
|
2019-02-11 09:07:26 +00:00 |
|
Tom Russell
|
731f299a18
|
Load open postcode data
|
2019-02-05 13:36:43 +00:00 |
|
Tom Russell
|
42f72bea9c
|
Add script to upload conservation areas
|
2019-01-20 15:29:54 +00:00 |
|
Tom Russell
|
d72dd90351
|
Comment sections in etl run script
|
2018-10-21 20:47:31 +01:00 |
|
Tom Russell
|
3441bf88e2
|
Update load_data to use API
|
2018-10-20 18:37:02 +01:00 |
|
Tom Russell
|
4415b21af5
|
Ensure index exists for uprn link
|
2018-10-04 21:13:25 +01:00 |
|
Tom Russell
|
a10e4bf5c6
|
Update run_all etl script
|
2018-10-04 19:01:40 +01:00 |
|
Tom Russell
|
30086766db
|
Boundary file not needed in initial extraction
|
2018-10-04 19:01:17 +01:00 |
|
Tom Russell
|
20e4a73e73
|
Skip altering foreign key restrictions
|
2018-10-04 19:00:56 +01:00 |
|
Tom Russell
|
4c62da548b
|
Drop outside limit of boundary
|
2018-10-04 18:59:53 +01:00 |
|
Tom Russell
|
b180602a5b
|
Uncomment copy-uprns block
|
2018-10-03 20:10:47 +01:00 |
|
Tom Russell
|
b73fb7118e
|
Fix sed quoting
|
2018-10-03 20:10:27 +01:00 |
|
Tom Russell
|
f06b820d19
|
Remove clipsrc for speedup
|
2018-10-03 20:10:16 +01:00 |
|
Tom Russell
|
4696e3e079
|
Update etl to load UPRNs to table
|
2018-10-02 21:12:46 +01:00 |
|
Tom Russell
|
79724cc449
|
Copy from stdin (cat user-accessible file) when loading geometries
|
2018-09-30 21:23:19 +01:00 |
|
Tom Russell
|
2a1902f6ce
|
Update ETL docs
|
2018-09-29 18:29:57 +01:00 |
|
Tom Russell
|
73aa3df290
|
Clip OSMM to GLA on extraction
|
2018-09-27 21:37:47 +01:00 |
|
Tom Russell
|
322a976f7a
|
Fix get_test_polygons script
|
2018-09-25 22:01:09 +01:00 |
|
Tom Russell
|
342167f9c9
|
Update UPRN-load script
|
2018-09-25 21:47:58 +01:00 |
|
Tom Russell
|
d9797385d9
|
Use default pool size (CPU count) in ETL
|
2018-09-25 21:47:29 +01:00 |
|
Tom Russell
|
3f9c9f3221
|
Split indexing further, UPRN requires bigint
|
2018-09-25 21:46:22 +01:00 |
|
Tom Russell
|
bddd7e769f
|
Rename etl scripts
|
2018-09-25 20:46:16 +01:00 |
|
Tom Russell
|
c6b3d3d5ca
|
Extract and load addressbase/mastermap using id, gml, parallel
|
2018-09-25 19:20:41 +01:00 |
|
Tom Russell
|
181e850225
|
Parallel extract/filter OS data
|
2018-09-21 11:10:39 +01:00 |
|
Tom Russell
|
f6f7cc1341
|
Save test polygons projected
|
2018-09-10 10:44:09 +01:00 |
|
Tom Russell
|
204740e46e
|
Sketch matching data by best-intersection
|
2018-09-09 11:58:50 +01:00 |
|
Tom Russell
|
5695f2dc9d
|
Record python packages for etl
|
2018-09-09 11:34:37 +01:00 |
|
Tom Russell
|
54633d9e04
|
Create building-per-geometry
|
2018-09-09 11:32:27 +01:00 |
|
Tom Russell
|
3711fc5f80
|
Load test polygons
|
2018-09-09 11:32:12 +01:00 |
|
Tom Russell
|
981d3608a2
|
Mappings for Camden and Fitrovia CASA data
|
2018-08-01 17:06:00 +01:00 |
|
Tom Russell
|
69bb6c5790
|
Provide transform when joining data
|
2018-08-01 17:05:02 +01:00 |
|
Tom Russell
|
847eb75a70
|
Fix loading buildings
|
2018-08-01 15:48:31 +01:00 |
|
Tom Russell
|
990e4241cf
|
Sketch generic update from shapes
|
2018-08-01 14:17:10 +01:00 |
|
Tom Russell
|
1c30f4d436
|
Skip trying to join buildings/geometries up front
|
2018-08-01 13:50:08 +01:00 |
|
Tom Russell
|
c7b322f214
|
Skip duplicate geometries
|
2018-08-01 13:49:16 +01:00 |
|
Tom Russell
|
f6c8323bfa
|
Load buildings from CSV
|
2018-08-01 13:49:03 +01:00 |
|
Tom Russell
|
cd65a5aaab
|
Load from geojson to db
|
2018-08-01 13:12:56 +01:00 |
|
Tom Russell
|
39160d8a09
|
Join each polygon to UPRNs if possible
|
2018-07-27 17:47:22 +01:00 |
|
Tom Russell
|
92b81928ca
|
Rename etl scripts to reflect stages
|
2018-07-27 15:48:00 +01:00 |
|
Tom Russell
|
105271a1e8
|
Sketch loading Addressbase buildings to database
- TODO handle intersection with geometries
|
2018-07-17 09:10:24 +01:00 |
|
Tom Russell
|
9316cfb719
|
Load OSMM geometries to database
|
2018-07-17 09:09:13 +01:00 |
|
Tom Russell
|
62b65f44f9
|
Extract OS MasterMap shell
|
2018-05-30 20:18:37 +01:00 |
|
Tom Russell
|
dd72ccd108
|
Extract OS AddressBase to CSV
|
2018-05-30 18:35:15 +01:00 |
|
Tom Russell
|
7c97aba950
|
Script comments for OS data extract
|
2018-05-30 14:31:23 +01:00 |
|