Documentation for working with DataFrames

[stensmo]

It's currently a bit difficult to understand how you should create DataFrames from JSON. I made this example code, which you can use or modify to post as an example. The important things are the keyword arguments to parse, as well as the Table.dictrowtable call.

using HTTP
using JSON
using DataFrames
using Plots
using Statistics


# Download the JSON data
url = "https://raw.githubusercontent.com/altair-viz/vega_datasets/master/vega_datasets/_data/wheat.json"
response = HTTP.get(url)

# Check for successful request
if response.status == 200
    Parse the JSON data, note that you must specify that null should be interpreted as missing and inttype should be Float64, otherwise Ints and Floats can be mixed in the same column
    data = JSON.parse(String(response.body);  null=missing, inttype=Float64)

    # Convert JSON to DataFrame. The Tables.dictrowtable is necessary for any data which does not have fields for all data. In this example wages are not specified for years 1815 and 1820
    df = DataFrame(Tables.dictrowtable(data))

    # Assuming 'year' is a column in your data
    years = df.year
    wheat = df.wheat
    wages = df.wages

    # Calculate a sliding window mean (example window size 5)
    window_size = 5::Int64
    wheat_rolling = [mean(wheat[i:(i + window_size - 1)]) for i in 1:(length(wheat) - window_size + 1)]
    wages_rolling = [mean(wages[i:(i + window_size - 1)]) for i in 1:(length(wages) - window_size + 1)]
    rolling_years = years[window_size÷2:(length(years) - window_size÷2 - 1)]


    # Plotting
    thePlot=plot(rolling_years, wheat_rolling, label="Wheat (Rolling Mean)",xlabel="Year", ylabel="Value")
    plot!(rolling_years, wages_rolling, label="Wages (Rolling Mean)")
    display(thePlot)

else
println("Error downloading the file: ", response.status)
end