nba_api
nba_api copied to clipboard
swar
NBA API: timeout for /stats endpoints
Hello,
First, I cannot thank you enough for this wonderful library.
I believe this has already been addressed in previous issues, but still shooting my shot. I'm having an issue with the NBA API for an app deployed on Heroku:
- the requests to
/statsendpoints time out in production, while working locally - the requests to
/liveendpoints work fine
I'm aware that the NBA block their API for Heroku instances, but I wanted to know if you were able to find a long-term workaround for this.
I'm using the version 1.1.13 of the library.
Thanks in advance :)
Hi, basically i have the same issue but not with /stats but with 'PlayerCareerStats' endpoint. I heard that upgrading versions to the newest would help, but i already have this done. I made request timeout longer to 90s but its still Read timed out situation every 9/10 situations.
Hi, basically i have the same issue but not with /stats but with 'PlayerCareerStats' endpoint. I heard that upgrading versions to the newest would help, but i already have this done. I made request timeout longer to 90s but its still Read timed out situation every 9/10 situations.
Yeah exactly :/
Same here. I'm trying to access the traditional boxscore endpoint and this happened to me.
I believe the header requirements by nba.com/stats has changed. The parameters are often the same with a few different API changes, e.g. boxscoreadvancedv2 => boxscoreadvancedv3 (version 3)
https://stats.nba.com/stats/boxscoreadvancedv3?
GameID=0022200701&LeagueID=00&endPeriod=0&endRange=28800&rangeType=0&startPeriod=0&startRange=0
But the parameters are the same, but doesn't work, gets a timeout issue. Which likely suggests the request header is an issue. I've tried 12 different endpoints and all get the same error. I'm using my laptop, so not a chunk of blocked AWS or Azure resources (which is a common issue with scraping).
When visiting the webpage directly and using the inspect element, I find the API and the Header. I then put those into code. But still having issues. Makes me wonder if need to capture the Response Cookie??
Here's an endpoint that is taken right from the inspector:
https://stats.nba.com/stats/leaguedashplayerptshot?CloseDefDistRange=&College=&Conference=&Country=&DateFrom=&DateTo=&Division=&DraftPick=&DraftYear=&DribbleRange=&GameScope=&GameSegment=&GeneralRange=Overall&Height=&LastNGames=0&LeagueID=00&Location=&Month=0&OpponentTeamID=0&Outcome=&PORound=0&PaceAdjust=N&PerMode=Totals&Period=0&PlayerExperience=&PlayerPosition=&PlusMinus=N&Rank=N&Season=2022-23&SeasonSegment=&SeasonType=Regular%20Season&ShotClockRange=&ShotDistRange=&StarterBench=&TeamID=0&TouchTimeRange=&VsConference=&VsDivision=&Weight=
Put in header form:
headers = {
'Host': 'stats.nba.com',
'Connection': 'keep-alive',
'Accept': 'application/json, text/plain, */*',
'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0.0.0 Safari/537.36',
'Origin': 'https://www.nba.com',
'Referer': 'https://www.nba.com/',
'Accept-Encoding': 'gzip, deflate, br',
'Accept-Language': 'en-US,en;q=0.5',
'Sec-Fetch-Dest': 'empty',
'Sec-Fetch-Mode': 'cors',
'Sec-Fetch-Site': 'same-site',
'Sec-GPC': '1'
}
This is the error message:
JSONDecodeError: Expecting value: line 1 column 1 (char 0)
During handling of the above exception, another exception occurred:
JSONDecodeError Traceback (most recent call last)
Input In [13], in <cell line: 22>()
50 url = f"{BASE_URL}{API_ENDPOINT}?{urlencode(params)}"
51 print(url)
---> 52 r = requests.get(url, headers=headers).json()
54 # names directory for file, placed file on given features and then checks for/creates director
55 PATH = NBA_DIR / f"{API_ENDPOINT}/{Season}/{SeasonType}/"
File ~/.pyenv/versions/3.10.2/lib/python3.10/site-packages/requests/models.py:917, in Response.json(self, **kwargs)
915 raise RequestsJSONDecodeError(e.message)
916 else:
--> 917 raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
JSONDecodeError: [Errno Expecting value] upstream request timeout: 0
Anyone else had success?
I've also seen a 504 response, again this is using local [Mac], Jupyter Notebook Python requests package. But in actually going to the nba.com website, seeing the API endpoint in the Network traffic, it's the same. So something to do with a cookie and/or header is my guess.
Here I get a few alternate 200 and 504s.
7 of 751 games
https://stats.nba.com/stats/boxscoretraditionalv2?EndPeriod=10&EndRange=43800&GameID=0022200294&RangeType=0&Season=2022-23&SeasonType=Regular+Season&StartPeriod=1&StartRange=0
200
<Response [200]>
Slept: 1.76 seconds
8 of 751 games
https://stats.nba.com/stats/boxscoretraditionalv2?EndPeriod=10&EndRange=43800&GameID=0022200142&RangeType=0&Season=2022-23&SeasonType=Regular+Season&StartPeriod=1&StartRange=0
504
<Response [504]>
Then interrupting and trying again, I get a 504 and kills it right away:
Games: 751
1 of 751 games
https://stats.nba.com/stats/boxscoretraditionalv2?EndPeriod=10&EndRange=43800&GameID=0022200657&RangeType=0&Season=2022-23&SeasonType=Regular+Season&StartPeriod=1&StartRange=0
504
---------------------------------------------------------------------------
JSONDecodeError Traceback (most recent call last)
File ~/.pyenv/versions/3.10.2/lib/python3.10/site-packages/requests/models.py:910, in Response.json(self, **kwargs)
909 try:
--> 910 return complexjson.loads(self.text, **kwargs)
911 except JSONDecodeError as e:
912 # Catch JSON-related errors and raise as requests.JSONDecodeError
913 # This aliases json.JSONDecodeError and simplejson.JSONDecodeError
File ~/.pyenv/versions/3.10.2/lib/python3.10/json/__init__.py:346, in loads(s, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, **kw)
343 if (cls is None and object_hook is None and
344 parse_int is None and parse_float is None and
345 parse_constant is None and object_pairs_hook is None and not kw):
--> 346 return _default_decoder.decode(s)
347 if cls is None:
File ~/.pyenv/versions/3.10.2/lib/python3.10/json/decoder.py:337, in JSONDecoder.decode(self, s, _w)
333 """Return the Python representation of ``s`` (a ``str`` instance
334 containing a JSON document).
335
336 """
--> 337 obj, end = self.raw_decode(s, idx=_w(s, 0).end())
338 end = _w(s, end).end()
File ~/.pyenv/versions/3.10.2/lib/python3.10/json/decoder.py:355, in JSONDecoder.raw_decode(self, s, idx)
354 except StopIteration as err:
--> 355 raise JSONDecodeError("Expecting value", s, err.value) from None
356 return obj, end
JSONDecodeError: Expecting value: line 1 column 1 (char 0)
During handling of the above exception, another exception occurred:
Edit/Update:
I put a sleeper of > 3.5 seconds and made sure to have the latest endpoint, e.g. v3 for boxscoretraditionalv3 and it seems to be working perfectly.
Games: 751
1 of 751 games
https://stats.nba.com/stats/boxscoretraditionalv3?EndPeriod=10&EndRange=43800&GameID=0022200657&RangeType=0&Season=2022-23&SeasonType=Regular+Season&StartPeriod=1&StartRange=0
200
Slept: 3.91 seconds
2 of 751 games
https://stats.nba.com/stats/boxscoretraditionalv3?EndPeriod=10&EndRange=43800&GameID=0022200641&RangeType=0&Season=2022-23&SeasonType=Regular+Season&StartPeriod=1&StartRange=0
200
Slept: 3.52 seconds
3 of 751 games
https://stats.nba.com/stats/boxscoretraditionalv3?EndPeriod=10&EndRange=43800&GameID=0022200240&RangeType=0&Season=2022-23&SeasonType=Regular+Season&StartPeriod=1&StartRange=0
200
Slept: 3.66 seconds
4 of 751 games
https://stats.nba.com/stats/boxscoretraditionalv3?EndPeriod=10&EndRange=43800&GameID=0022200102&RangeType=0&Season=2022-23&SeasonType=Regular+Season&StartPeriod=1&StartRange=0
200
Slept: 3.82 seconds
5 of 751 games
https://stats.nba.com/stats/boxscoretraditionalv3?EndPeriod=10&EndRange=43800&GameID=0022200358&RangeType=0&Season=2022-23&SeasonType=Regular+Season&StartPeriod=1&StartRange=0
200
Slept: 3.57 seconds
6 of 751 games
https://stats.nba.com/stats/boxscoretraditionalv3?EndPeriod=10&EndRange=43800&GameID=0022200308&RangeType=0&Season=2022-23&SeasonType=Regular+Season&StartPeriod=1&StartRange=0
200
Slept: 3.56 seconds
7 of 751 games
https://stats.nba.com/stats/boxscoretraditionalv3?EndPeriod=10&EndRange=43800&GameID=0022200294&RangeType=0&Season=2022-23&SeasonType=Regular+Season&StartPeriod=1&StartRange=0
200
Hi, I have the same issue.
- the requests to /stats endpoints (for example 'https://stats.nba.com/stats/commonteamroster') time out in production (AWS), while working locally.
- the requests to /live endpoints work fine (for example 'https://cdn.nba.com/static/json/liveData/scoreboard/todaysScoreboard_00.json')
I am using version 1.1.14 of the library.
I tried to use different headers with no success.
Has anyone resolved this issue? Thanks.
Since AWS/GCP/Heroku (maybe others too) IP addresses are blocked by NBA, a solution doesn't seem possible using Headers. Only possible solution as far as I know is using proxies while making requests. If you already have a VPN provider, you just need to dig some documentation to fetch proxy endpoint information. That way I was able to use /stats endpoints on AWS Lambda.
Some endpoints were working for me today but stopped after a bit. I turned on my VPN and it worked again. Seems like my IP gets blocked after a while which is weird since I didn't use it extensively.
does anyone know how to use proxies? I looked up free proxies yesterday but none of them worked. I passed in "{proxy}:{port}" in the proxy param for the call but connection still failed
does anyone know how to use proxies? I looked up free proxies yesterday but none of them worked. I passed in "{proxy}:{port}" in the proxy param for the call but connection still failed
Its so difficult to find a free working proxy from the web. You can check out paid proxy services
has anyone resolved with streamlit by any chance? it works fine on local but it errors out when i try to deploy
requests.exceptions.ReadTimeout: This app has encountered an error. The original error message is redacted to prevent data leaks. Full error details have been recorded in the logs (if you're on Streamlit Cloud, click on 'Manage app' in the lower right of your app).
Traceback:
File "/home/adminuser/venv/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 541, in _run_script
exec(code, module.__dict__)
File "/mount/src/nba_stat/main.py", line 39, in <module>
career_stat = playercareerstats.PlayerCareerStats(player_id=player_id).get_data_frames()
File "/home/adminuser/venv/lib/python3.9/site-packages/nba_api/stats/endpoints/playercareerstats.py", line 34, in __init__
self.get_request()
File "/home/adminuser/venv/lib/python3.9/site-packages/nba_api/stats/endpoints/playercareerstats.py", line 37, in get_request
self.nba_response = NBAStatsHTTP().send_api_request(
File "/home/adminuser/venv/lib/python3.9/site-packages/nba_api/library/http.py", line 130, in send_api_request
response = requests.get(url=base_url, params=parameters, headers=request_headers, proxies=proxies, timeout=timeout)
File "/home/adminuser/venv/lib/python3.9/site-packages/requests/api.py", line 73, in get
return request("get", url, params=params, **kwargs)
File "/home/adminuser/venv/lib/python3.9/site-packages/requests/api.py", line 59, in request
return session.request(method=method, url=url, **kwargs)
File "/home/adminuser/venv/lib/python3.9/site-packages/requests/sessions.py", line 589, in request
resp = self.send(prep, **send_kwargs)
File "/home/adminuser/venv/lib/python3.9/site-packages/requests/sessions.py", line 703, in send
r = adapter.send(request, **kwargs)
File "/home/adminuser/venv/lib/python3.9/site-packages/requests/adapters.py", line 532, in send
raise ReadTimeout(e, request=request)
I had a mix experience with Streamlit, sometimes it works, but some others it doesn't. I work mainly with the endpoints leaguegamefinder.LeagueGameFinder() and boxscoretraditionalv2.BoxScoreTraditionalV2().
Now, I believe I'm facing this issue with GitHub Actions. I have a scheduled action to fetch recent games using the endpoint leaguegamefinder.LeagueGameFinder(). I'm testing it to fetch info from only 6 games (a 6-row dataframe), but I'm getting the following error:
raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='stats.nba.com', port=443): Read timed out. (read timeout=30)
This is the full trace:
Traceback (most recent call last):
File "/opt/hostedtoolcache/Python/3.10.11/x64/lib/python3.10/site-packages/urllib3/connectionpool.py", line 467, in _make_request
six.raise_from(e, None)
File "<string>", line 3, in raise_from
File "/opt/hostedtoolcache/Python/3.10.11/x64/lib/python3.10/site-packages/urllib3/connectionpool.py", line 462, in _make_request
httplib_response = conn.getresponse()
File "/opt/hostedtoolcache/Python/3.10.11/x64/lib/python3.10/http/client.py", line 1375, in getresponse
response.begin()
File "/opt/hostedtoolcache/Python/3.10.11/x64/lib/python3.10/http/client.py", line 318, in begin
version, status, reason = self._read_status()
File "/opt/hostedtoolcache/Python/3.10.11/x64/lib/python3.10/http/client.py", line 279, in _read_status
line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
File "/opt/hostedtoolcache/Python/3.10.11/x64/lib/python3.10/socket.py", line 705, in readinto
return self._sock.recv_into(b)
File "/opt/hostedtoolcache/Python/3.10.11/x64/lib/python3.10/ssl.py", line 1274, in recv_into
return self.read(nbytes, buffer)
File "/opt/hostedtoolcache/Python/3.10.11/x64/lib/python3.10/ssl.py", line 1130, in read
return self._sslobj.read(len, buffer)
TimeoutError: The read operation timed out
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/hostedtoolcache/Python/3.10.11/x64/lib/python3.10/site-packages/requests/adapters.py", line 486, in send
resp = conn.urlopen(
File "/opt/hostedtoolcache/Python/3.10.11/x64/lib/python3.10/site-packages/urllib3/connectionpool.py", line 799, in urlopen
retries = retries.increment(
File "/opt/hostedtoolcache/Python/3.10.11/x64/lib/python3.10/site-packages/urllib3/util/retry.py", line 550, in increment
raise six.reraise(type(error), error, _stacktrace)
File "/opt/hostedtoolcache/Python/3.10.11/x64/lib/python3.10/site-packages/urllib3/packages/six.py", line 770, in reraise
raise value
File "/opt/hostedtoolcache/Python/3.10.11/x64/lib/python3.10/site-packages/urllib3/connectionpool.py", line 715, in urlopen
httplib_response = self._make_request(
File "/opt/hostedtoolcache/Python/3.10.11/x64/lib/python3.10/site-packages/urllib3/connectionpool.py", line 469, in _make_request
self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
File "/opt/hostedtoolcache/Python/3.10.11/x64/lib/python3.10/site-packages/urllib3/connectionpool.py", line 358, in _raise_timeout
raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='stats.nba.com', port=443): Read timed out. (read timeout=30)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/runner/work/***/***/src/fetch_data.py", line 326, in <module>
fetch_recent_games()
File "/home/runner/work/***/***/src/fetch_data.py", line 260, in fetch_recent_games
new_data_regular_season = leaguegamefinder.LeagueGameFinder(
File "/opt/hostedtoolcache/Python/3.10.11/x64/lib/python3.10/site-packages/nba_api/stats/endpoints/leaguegamefinder.py", line [20](https://github.com/jacasta2/nba_analysis/actions/runs/8010834757/job/21882723941#step:5:21)4, in __init__
self.get_request()
File "/opt/hostedtoolcache/Python/3.10.11/x64/lib/python3.10/site-packages/nba_api/stats/endpoints/leaguegamefinder.py", line 207, in get_request
self.nba_response = NBAStatsHTTP().send_api_request(
File "/opt/hostedtoolcache/Python/3.10.11/x64/lib/python3.10/site-packages/nba_api/library/http.py", line 1[30](https://github.com/jacasta2/nba_analysis/actions/runs/8010834757/job/21882723941#step:5:31), in send_api_request
response = requests.get(url=base_url, params=parameters, headers=request_headers, proxies=proxies, timeout=timeout)
File "/opt/hostedtoolcache/Python/3.10.11/x64/lib/python3.10/site-packages/requests/api.py", line 73, in get
return request("get", url, params=params, **kwargs)
File "/opt/hostedtoolcache/Python/3.10.11/x64/lib/python3.10/site-packages/requests/api.py", line 59, in request
return session.request(method=method, url=url, **kwargs)
File "/opt/hostedtoolcache/Python/3.10.11/x64/lib/python3.10/site-packages/requests/sessions.py", line 589, in request
resp = self.send(prep, **send_kwargs)
File "/opt/hostedtoolcache/Python/3.10.11/x64/lib/python3.10/site-packages/requests/sessions.py", line 703, in send
r = adapter.send(request, **kwargs)
File "/opt/hostedtoolcache/Python/3.10.11/x64/lib/python3.10/site-packages/requests/adapters.py", line 5[32](https://github.com/jacasta2/nba_analysis/actions/runs/8010834757/job/21882723941#step:5:33), in send
raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='stats.nba.com', port=4[43](https://github.com/jacasta2/nba_analysis/actions/runs/8010834757/job/21882723941#step:5:44)): Read timed out. (read timeout=30)
What a buzzkiller!
