Commit Graph

35 Commits

Author SHA1 Message Date
9f53c66975 adding latest beam pipeline code for dataflow 2021-09-26 15:57:00 +01:00
e6ec110d54 updating download data script for new location in GCS 2021-09-26 14:56:53 +01:00
83807616e0 Merge branch 'wip/pathlib' into develop 2021-09-26 14:56:21 +01:00
7f874fa6f6 updating README.md 2021-09-26 14:56:01 +01:00
b8a997084d removing requirements.txt for documentation 2021-09-26 14:55:48 +01:00
c4e81065b1 adding pyenv 3.7.9 2021-09-26 14:55:05 +01:00
62bd0196ad removing shard_name_template from output file 2021-09-26 06:11:42 +01:00
7f9b7e4bfd moving inputs/outputs to use pathlib 2021-09-26 06:03:55 +01:00
7962f40e32 adding initial docs 2021-09-26 01:34:06 +01:00
2a43ea1946 adding download script for data set 2021-09-26 01:18:23 +01:00
07d176be79 updating .gitignore 2021-09-26 01:10:53 +01:00
f804e85cc3 updating .gitignore 2021-09-26 01:10:32 +01:00
9fdc6dce05 migrate beam pipeline to main.py 2021-09-26 01:10:09 +01:00
54cf5e3e36 updating prospector.yaml 2021-09-26 01:09:48 +01:00
2e42a453b0 adding latest beam pipeline code 2021-09-25 22:15:58 +01:00
adfbd8e93d updating prospector.yaml 2021-09-25 22:15:46 +01:00
1bd54f188d updating folder structure for data input/output 2021-09-25 22:15:39 +01:00
a7c52b1085 updating .gitignore 2021-09-25 22:15:10 +01:00
24420c8935 adding latest beam pipeline code 2021-09-25 18:25:08 +01:00
44f346deff Merge remote-tracking branch 'origin/develop' into develop 2021-09-25 16:48:04 +01:00
a37e7817c3 adding latest beam pipeline code 2021-09-25 16:47:52 +01:00
539e4c7786 adding latesty beam pipeline code 2021-09-25 16:47:17 +01:00
214ce77d8f adding debug.py 2021-09-25 16:47:08 +01:00
47a4ac4bc3 adding latest beam pipeline code 2021-09-25 01:44:37 +01:00
aa61ea9c57 adding latest beam pipeline code 2021-09-25 00:48:41 +01:00
94cc22a385 adding data exploration report + code 2021-09-25 00:48:34 +01:00
a05182892a adding documentation 2021-09-25 00:48:13 +01:00
c38a10ca2f updating beam to install gcp extras 2021-09-25 00:48:04 +01:00
ab993fc030 adding prospector.yaml 2021-09-25 00:47:49 +01:00
9cf5662600 updating notes 2021-09-24 16:52:40 +01:00
0f262daf39 adding report.py 2021-09-24 16:52:34 +01:00
972d0a852a updating dev dependencies 2021-09-24 16:52:25 +01:00
5301d5ff04 updating .gitignore 2021-09-24 16:51:56 +01:00
6db2fa59b9 adding initial skeleton 2021-09-24 16:01:03 +01:00
dtomlinson91
8a22bfebe1 Initial commit 2021-09-24 15:52:38 +01:00