mirror of
https://github.com/dtomlinson91/street_group_tech_test
synced 2025-12-22 11:55:45 +00:00
25 lines
786 B
Markdown
25 lines
786 B
Markdown
# DataFlow
|
|
|
|
<https://cloud.google.com/dataflow/docs/quickstarts/quickstart-python>
|
|
|
|
Export env variable:
|
|
|
|
`export GOOGLE_APPLICATION_CREDENTIALS="/home/dtomlinson/git-repos/work/street_group/street_group_tech_test/street-group-0c490d23a9d0.json"`
|
|
|
|
Run the pipeline:
|
|
|
|
python -m analyse_properties.main \
|
|
--region europe-west2 \
|
|
--input gs://street-group-technical-test-dmot/input/pp-monthly-update-new-version.csv \
|
|
--output gs://street-group-technical-test-dmot/output/pp-monthly-update-new-version \
|
|
--runner DataflowRunner \
|
|
--project street-group \
|
|
--temp_location gs://street-group-technical-test-dmot/tmp
|
|
|
|
|
|
## Errors
|
|
|
|
Unsubscriptable error on window:
|
|
|
|
<https://stackoverflow.com/questions/42276520/what-does-object-of-type-unwindowedvalues-has-no-len-mean>
|