Commit Graph
40 Commits
Author SHA1 Message Date
dtomlinson 4e3771c728 adding latest beam pipeline code for dataflow 2021-09-26 17:15:35 +01:00
dtomlinson 8856a9763f updating prospector.yaml 2021-09-26 17:15:24 +01:00
dtomlinson fded858932 moving dataflow notes 2021-09-26 17:15:17 +01:00
dtomlinson bb71d55f8c adding latest beam pipeline code for dataflow 2021-09-26 16:23:19 +01:00
dtomlinson 8047b5ced4 adding latest beam pipeline code for dataflow 2021-09-26 16:16:58 +01:00
dtomlinson 9f53c66975 adding latest beam pipeline code for dataflow 2021-09-26 15:57:00 +01:00
dtomlinson e6ec110d54 updating download data script for new location in GCS 2021-09-26 14:56:53 +01:00
dtomlinson 83807616e0 Merge branch 'wip/pathlib' into develop 2021-09-26 14:56:21 +01:00
dtomlinson 7f874fa6f6 updating README.md 2021-09-26 14:56:01 +01:00
dtomlinson b8a997084d removing requirements.txt for documentation 2021-09-26 14:55:48 +01:00
dtomlinson c4e81065b1 adding pyenv 3.7.9 2021-09-26 14:55:05 +01:00
dtomlinson 62bd0196ad removing shard_name_template from output file 2021-09-26 06:11:42 +01:00
dtomlinson 7f9b7e4bfd moving inputs/outputs to use pathlib 2021-09-26 06:03:55 +01:00
dtomlinson 7962f40e32 adding initial docs 2021-09-26 01:34:06 +01:00
dtomlinson 2a43ea1946 adding download script for data set 2021-09-26 01:18:23 +01:00
dtomlinson 07d176be79 updating .gitignore 2021-09-26 01:10:53 +01:00
dtomlinson f804e85cc3 updating .gitignore 2021-09-26 01:10:32 +01:00
dtomlinson 9fdc6dce05 migrate beam pipeline to main.py 2021-09-26 01:10:09 +01:00
dtomlinson 54cf5e3e36 updating prospector.yaml 2021-09-26 01:09:48 +01:00
dtomlinson 2e42a453b0 adding latest beam pipeline code 2021-09-25 22:15:58 +01:00
dtomlinson adfbd8e93d updating prospector.yaml 2021-09-25 22:15:46 +01:00
dtomlinson 1bd54f188d updating folder structure for data input/output 2021-09-25 22:15:39 +01:00
dtomlinson a7c52b1085 updating .gitignore 2021-09-25 22:15:10 +01:00
dtomlinson 24420c8935 adding latest beam pipeline code 2021-09-25 18:25:08 +01:00
dtomlinson 44f346deff Merge remote-tracking branch 'origin/develop' into develop 2021-09-25 16:48:04 +01:00
dtomlinson a37e7817c3 adding latest beam pipeline code 2021-09-25 16:47:52 +01:00
dtomlinson 539e4c7786 adding latesty beam pipeline code 2021-09-25 16:47:17 +01:00
dtomlinson 214ce77d8f adding debug.py 2021-09-25 16:47:08 +01:00
dtomlinson 47a4ac4bc3 adding latest beam pipeline code 2021-09-25 01:44:37 +01:00
dtomlinson aa61ea9c57 adding latest beam pipeline code 2021-09-25 00:48:41 +01:00
dtomlinson 94cc22a385 adding data exploration report + code 2021-09-25 00:48:34 +01:00
dtomlinson a05182892a adding documentation 2021-09-25 00:48:13 +01:00
dtomlinson c38a10ca2f updating beam to install gcp extras 2021-09-25 00:48:04 +01:00
dtomlinson ab993fc030 adding prospector.yaml 2021-09-25 00:47:49 +01:00
dtomlinson 9cf5662600 updating notes 2021-09-24 16:52:40 +01:00
dtomlinson 0f262daf39 adding report.py 2021-09-24 16:52:34 +01:00
dtomlinson 972d0a852a updating dev dependencies 2021-09-24 16:52:25 +01:00
dtomlinson 5301d5ff04 updating .gitignore 2021-09-24 16:51:56 +01:00
dtomlinson 6db2fa59b9 adding initial skeleton 2021-09-24 16:01:03 +01:00
dtomlinson91 8a22bfebe1 Initial commit 2021-09-24 15:52:38 +01:00