colouring-montreal

History

Mateusz Konieczny e7f67cbe8f treat invalid future dates as invalid Brent-87_0946 has "valid_date": "23/04/9187" yes, they apparently recorded filing in Anno Dominin 9187		2022-11-08 23:16:03 +01:00
..
.gitignore	Display official planning data.	2022-10-05 19:52:14 +02:00
obtain_livestream_data_and_load_into_database.py	treat invalid future dates as invalid	2022-11-08 23:16:03 +01:00
README.MD	fix typo	2022-11-05 08:32:29 +01:00
requirements.txt	change data source	2022-10-31 10:09:12 +01:00

README.MD

Following instructions assume that code is placed within ~/colouring-london/etl/planning_data/

To install necessary dependencies use cd ~/colouring-london/etl/planning_data/ && pip3 install -r requirements.txt

Following scripts should be scheduled to run regularly to load livestream data into database.

# querying API to obtain data & loading data into Colouring database
python3 obtain_livestream_data_and_load_into_database.py

# removing tile cache for planning_applications_status layer - note that location of cache depends on your configuration
rm /srv/colouring-london/tilecache/planning_applications_status/* -rf

As loading into databases expects environment variables to be set, one option to actually schedule it in a cron is something like

export $(cat ~/scripts/.env | xargs) && /usr/bin/python3 ~/colouring-london/etl/planning_data/obtain_livestream_data_and_load_into_database.py

with

~/scripts/.env

being in following format

PGHOST=localhost
PGDATABASE=colouringlondondb
PGUSER=cldbadmin
PGPASSWORD=actualpassword
PLANNNING_DATA_API_ALLOW_REQUEST_CODE=requestcode