Commit Graph

59 Commits

Author SHA1 Message Date
Tom Russell
af54a67051 Fix lints 2023-08-15 14:10:19 +01:00
Tom Russell
5378fac326 Bump python for CI, install psycopg2 from wheel 2023-08-15 14:10:08 +01:00
dependabot[bot]
854fd46520
Bump requests from 2.23.0 to 2.31.0 in /etl
Bumps [requests](https://github.com/psf/requests) from 2.23.0 to 2.31.0.
- [Release notes](https://github.com/psf/requests/releases)
- [Changelog](https://github.com/psf/requests/blob/main/HISTORY.md)
- [Commits](https://github.com/psf/requests/compare/v2.23.0...v2.31.0)

---
updated-dependencies:
- dependency-name: requests
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-05-22 23:31:22 +00:00
Mateusz Konieczny
9892ddd386
describe planning data script in context of core 2023-02-19 07:36:37 +01:00
Mateusz Konieczny
a022aa6f4d change DataHub to Datahub 2023-01-23 06:49:45 +01:00
Mateusz Konieczny
7d7edf6df3 add data source link again 2023-01-19 14:41:23 +01:00
Mateusz Konieczny
93295624a5 fix Hackney links
application_id source was changed and it broke this hidden internal use of it - here we need borough identifier

fixes #1047 and repairs dffa4e244b
2023-01-19 14:40:16 +01:00
Mateusz Konieczny
be23cf2fb6 add missing thes, fixes #1045 2023-01-19 14:40:16 +01:00
Mateusz Konieczny
5c3139ff6f drop unwanted layers 2022-12-02 15:28:50 +01:00
Mateusz Konieczny
dc5e00f7e6 try more clear logging 2022-11-23 10:50:55 +01:00
Mateusz Konieczny
930fb11e6a show address data in planning data green box 2022-11-23 08:59:10 +01:00
Mateusz Konieczny
58e6625790 apply some pep8 rules to python script 2022-11-13 13:36:04 +01:00
Mateusz Konieczny
9512d30c28 add status explanation note
this allow to handle OBS statuses, also done in this commit
2022-11-13 09:01:38 +01:00
Mateusz Konieczny
dffa4e244b import id more fitting for display 2022-11-12 18:48:58 +01:00
Mateusz Konieczny
e2a2516619 fix missing newline 2022-11-10 10:21:57 +01:00
Mateusz Konieczny
45b8aeea81 remove duplicative handling of status
Unknown and Lapse are aliased in steps before
2022-11-10 10:19:45 +01:00
Mateusz Konieczny
8ae0c0c5f9 better protected against database failing
entries with long unaliased status could cause problem in future with new aliases
2022-11-10 10:18:08 +01:00
Mateusz Konieczny
5ed2d419a9 reorder imports like linter wanted 2022-11-10 10:17:33 +01:00
Mateusz Konieczny
d544b5228d apply PEP8 standards 2022-11-10 08:29:14 +01:00
Mateusz Konieczny
ffe43dbd38 show data that caused failure 2022-11-08 23:16:03 +01:00
Mateusz Konieczny
63ec84e9b2 handle dates in passed data on reporting errors
json.dumps fails on date objects
2022-11-08 23:16:03 +01:00
Mateusz Konieczny
e7f67cbe8f treat invalid future dates as invalid
Brent-87_0946 has "valid_date": "23/04/9187"
yes, they apparently recorded filing in Anno Dominin 9187
2022-11-08 23:16:03 +01:00
Mateusz Konieczny
da4b198555 remove duplication 2022-11-08 23:08:46 +01:00
Mateusz Konieczny
b4b17c7c48 date cacing hack 2022-11-08 16:23:38 +01:00
Mateusz Konieczny
8a037b2ebc delete no longer used code 2022-11-08 15:59:47 +01:00
Mateusz Konieczny
4fba0a9399 multiline for code readability 2022-11-08 12:35:37 +01:00
Mateusz Konieczny
48b9cec85c drop dead code 2022-11-08 12:35:37 +01:00
Mateusz Konieczny
a73d463b48 turn dates into strings only at the end 2022-11-08 12:35:37 +01:00
Mateusz Konieczny
1be12dcb99 links for portals in more boroughs 2022-11-07 14:56:13 +01:00
Mateusz Konieczny
23133c387f try to help with links 2022-11-07 14:32:46 +01:00
Mateusz Konieczny
f1c81b1778 more aliasing 2022-11-07 10:51:09 +01:00
Mateusz Konieczny
ba605dbb5f in case of error during data load just fail 2022-11-05 10:06:49 +01:00
Mateusz Konieczny
5d3f374cf7 record and show raw status from data 2022-11-05 10:06:49 +01:00
Mateusz Konieczny
2ed84f1148 treat 'Approval' as synonym for 'Approved' 2022-11-05 08:42:17 +01:00
Mateusz Konieczny
d67abd4cc8 treat Dismissed as synonym to Refused 2022-11-05 08:34:35 +01:00
Mateusz Konieczny
a236729e9b treat Allowed as synonym to Approved 2022-11-05 08:34:06 +01:00
Mateusz Konieczny
90d2715f4c fix typo 2022-11-05 08:32:29 +01:00
Mateusz Konieczny
bef2beb5c7 fix typo 2022-11-05 08:32:16 +01:00
Mateusz Konieczny
b8adaecf39 special handling of Hackney links 2022-11-01 20:19:15 +01:00
Mateusz Konieczny
ae56933462 entries with decision_date set are never aliased to status=Submitted 2022-11-01 20:18:57 +01:00
Mateusz Konieczny
32076a9012 drop linking as we have no link to share 2022-11-01 20:18:21 +01:00
Mateusz Konieczny
8521f9b4f1 stop malforming links 2022-10-31 10:33:54 +01:00
Mateusz Konieczny
89d4b57fde document remaining decision 2022-10-31 10:32:19 +01:00
Mateusz Konieczny
0f689217c3 change data source 2022-10-31 10:09:12 +01:00
Mateusz Konieczny
205580d7fe better display as unusual data in planning view
applications with unandled statuses are shown
not available application id is not shown as unknown anymore
2022-10-05 19:56:25 +02:00
Mateusz Konieczny
90dae39f12 tweak wording of disclaimer 2022-10-05 19:56:25 +02:00
Mateusz Konieczny
bfe8720f09 explicitly skip more confusing statuses 2022-10-05 19:54:39 +02:00
Mateusz Konieczny
71b67d9890 still skip unexpected statuses 2022-10-05 19:54:39 +02:00
Mateusz Konieczny
1fce51c104 stop treating unexpected status as error 2022-10-05 19:54:39 +02:00
Mateusz Konieczny
9c19ad24ba one more status 2022-10-05 19:54:39 +02:00