reqHistoricalNews date range filter does not work

[Fredrik-C]

The output is identical for these two requests:

contract = Stock('AAPL', 'SMART', 'USD')
ib.qualifyContracts(contract)

start = datetime.datetime.strptime('2025-01-14 09:30:00', '%Y-%m-%d %H:%M:%S')
end = datetime.datetime.strptime('2025-01-30 09:30:00', '%Y-%m-%d %H:%M:%S')
headlines = ib.reqHistoricalNews(conId=contract.conId, providerCodes=codes, startDateTime=start, endDateTime=end, totalResults=300)
print(f'{headlines[-1].time} : {headlines[0].time}')

start = datetime.datetime.strptime('2025-01-01 09:30:00', '%Y-%m-%d %H:%M:%S')
end = datetime.datetime.strptime('2025-01-14 09:30:00', '%Y-%m-%d %H:%M:%S')
headlines = ib.reqHistoricalNews(conId=contract.conId, providerCodes=codes, startDateTime=start, endDateTime=end, totalResults=300)
print(f'{headlines[-1].time} : {headlines[0].time}')

Output:

[NewsProvider(code='BRFG', name='Briefing.com General Market Columns'), NewsProvider(code='BRFUPDN', name='Briefing.com Analyst Actions'), NewsProvider(code='DJ-N', name='Dow Jones News Service'), NewsProvider(code='DJ-RTA', name='Dow Jones Real-Time News Asia Pacific'), NewsProvider(code='DJ-RTE', name='Dow Jones Real-Time News Europe'), NewsProvider(code='DJ-RTG', name='Dow Jones Real-Time News Global'), NewsProvider(code='DJ-RTPRO', name='Dow Jones Real-Time News Pro'), NewsProvider(code='DJNL', name='Dow Jones Newsletters')]
2025-01-07 14:59:00 : 2025-01-30 16:29:00
2025-01-07 14:59:00 : 2025-01-30 16:29:00

ib_async-1.0.3

[mattsta]

I've never used the reqHistoricalNews API, but one guess is maybe the time formats are being ignored?

The official documentation for reqHistoricalNews says the time format is (a bit weird):

The format is yyyy-MM-dd HH:mm:ss.0

So, I wonder if the ib_async built-in datetime object conversion isn't printing the correct date format.

If you already have date strings (as in the example above) you can use them directly without converting to datetime objects first, so you could add the extra .0 to the end and see if it helps the dates work?

As a test, you could use the datetime strings directly in the .0 API format then report back with testing both ways (datetime objects as timestamps and also just strings as timestamps) to figure out if the built-in reformatting is using the incorrect format for this API endpoint.

[Fredrik-C]

It seems that startDateTime & endDateTime is not working as a time span. If I specify a startDateTime I will get historical news from that date and backwards in history.

start = '2025-02-01 09:30:00'
end = '2025-01-01 09:30:00'
headlines = ib.reqHistoricalNews(conId=contract.conId, providerCodes=codes, startDateTime=start, endDateTime=end, totalResults=300)
print(f'{headlines[-1].time} : {headlines[0].time} : {len(headlines)}')

start = '2025-01-01 09:30:00'
end = '2024-12-14 09:30:00'
headlines2 = ib.reqHistoricalNews(conId=contract.conId, providerCodes=codes, startDateTime=start, endDateTime=end, totalResults=300)
print(f'{headlines2[-1].time} : {headlines2[0].time} : {len(headlines2)}')

2025-01-23 16:17:00 : 2025-02-01 07:32:00 : 300
2024-12-17 11:39:00 : 2025-01-01 00:05:00 : 300

One annoying thing however is that reqHistoricalNews has a hardcoded request timeout set to 4.

[mattsta]

Interesting edge case discovered!

So the code for creating date strings from datetime objects needs to be modified when using ib.reqHistoricalNews() calls. I'll make a note of that for the next update cycle (which is hopefully soon). It may be causing the backwards time problems? Somewhat uncertain, but I may be able to run some more tests against it to figure out all the different problems between formatting and time directions causing weird issues.

I'll also make a note to check the hard coded request timeout you mention. Sometimes old values like that are to hide other bugs, and sometimes they are just mistakes nobody has really noticed over time.

[BadPlayer555]

I also encountered this issue. The responses from reqHistoricalNews are also not within the date range and the number of articles are not approaching the totalResults (maximum 300).

So then I did some investigations and used the official python ibapi (not the wrapper), also the official C#, Java demo to test this function. They also return the same non-sense.

So I think the bug is actually on the server api's side.