Would anyone happent to have an example of integrating this with a csv target?

[Nishq022]

Hi, I'm trying to pull data from fb using taps and targets, and have been unsuccessful so far. I've tried the following:

~/.virtualenvs/tap-facebook/bin/tap-facebook --config config.json --catalog catalog.json | ~/.virtualenvs/target-csv/bin/target-csv >target_output.csv

~/.virtualenvs/tap-facebook/bin/tap-facebook -c config.json -s state.json --discover | ~/.virtualenvs/target-csv/bin/target-csv >target_output.csv

But each time I'm getting a blank output. My state.json file looks like this: {"ads":"2019-01-01T00:00:00Z", "adcreative":"2019-01-01T00:00:00Z", "ads_insights":"2019-01-01T00:00:00Z"}

I also have another version of tap-facebook installed on a different virtualenv, and simply use the cmd:

/venv/bin/tap-facebook -c config.json --discover > tap_output.json | ~/.virtualenvs/target-csv/bin/target-csv >target_output.csv

I get an output in tap_output.csv, which I'm guessing is the schema. But no output in target-csv, any idea how to get that working?

Would really appreciate any help!

[siefert]

I might be wrong but I don't believe that anything is being piped to target-csv in your case. That pipe is not piping anything as all output is being directed to 'tap_output.json'.

[kayosouza]

Do the following steps:

INSTALL TAP FACEBOOK

python3 -m venv ~/.virtualenvs/tap-facebook
source ~/.virtualenvs/tap-facebook/bin/activate
pip install tap-facebook
deactivate

INSTALL TARGET CSV

python3 -m venv ~/.virtualenvs/target-csv
source ~/.virtualenvs/target-csv/bin/activate
pip install target-csv
deactivate

CONFIG

{
  "ads": "2019-01-01T00:00:00Z",
  "adcreative": "2019-01-01T00:00:00Z",
  "ads_insights": "2019-01-01T00:00:00Z"
}

GET DATA ON FACEBOOK GRAPHQL API

1. QUERY ACCOUNT_ID

fields=client_ad_accounts{name,account_id}

RUNNING DISCOVERY MODE

~/.virtualenvs/tap-facebook/bin/tap-facebook -c tap_config.json --discover > catalog.json

choose the properties listed in catalog.json then paste it on properties.json, e.g:

{
  "breadcrumb": [],
  "metadata": {
      "table-key-properties": [
          "campaign_id",
          "adset_id",
          "ad_id",
          "date_start"
      ],
      "inclusion": "available",
      "selected": "true"
  }
},

RUNNING ETL

~/.virtualenvs/tap-facebook/bin/tap-facebook -c tap_config.json -p properties.json | ~/.virtualenvs/target-csv/bin/target-csv

[KobaKhit]

The output from tap-facebook has nested json objects which does not translate to csv well. As a result, if you feed it through the target-csv you will get misaligned columns.

[yogesh-rudder]

Could anyone please tell me how I can run the last command from @kayosouza 's answer on Windows system. command: ~/.virtualenvs/tap-facebook/bin/tap-facebook -c tap_config.json -p properties.json | ~/.virtualenvs/target-csv/bin/target-csv

[auyer]

I'll share how I use it with target-parquet (almost the same as csv). I run this is a docker container, In the

Dockerfile

... setup python & etc

# create 2 virtualenvs
RUN virtualenv -p python3 venv/tap-facebook
RUN virtualenv -p python3 venv/target-parquet 

RUN ./venv/tap-facebook/bin/pip install tap-facebook --use-feature=2020-resolver
RUN ./venv/target-parquet/bin/pip install target-parquet # (i actually us a custom version that is not in pip yet)

...

CMD ["bash", "run.sh"]

run.sh script (that will be called by the dockerfile in runtime

... download state and config 

./venv/tap-facebook/bin/tap-facebook -c ./config.json -s ./state.json -p ./selected.catalog.json | tee -a >( gzip -9 > ./raw/"$(date +%FT%T).log" )  | ./venv/target-parquet/bin/target-parquet -c ./parquet.config.json >> ./output/state.json

## re-write state file. The state is actually in the last line.
tail -1 ./output/state.json > state.json.tmp && mv state.json.tmp ./output/state.json

... upload data

This should be enough for most setups. If you are not getting any output, pipe the output of the tap-facebook to a file, and check if there are any "record" messages. Check the singer spec for more details.

Could anyone please tell me how I can run the last command from @kayosouza 's answer on Windows system.

I saw multiple users having issues to run these taps in OSs different than Linux. I would recommend wither using Linux directly or Docker.