News Date Issue

[scottstables]

using the example:

finviz.get_news('AAPL')

the following is returned:

ValueError: time data '\r\n Today 11:17AM\r\n ' does not match format '%b-%d-%y %I:%M%p'

Did something change on the FinViz end in the last week ?

[AGG2017]

Instead of the real date they put Today and fast fix at line 113 in main_func.py should be:

            if 'Today' in raw_timestamp:
                parsed_timestamp = datetime.now()
            else:
                parsed_timestamp = datetime.strptime(raw_timestamp, "%b-%d-%y %I:%M%p")
            date = parsed_timestamp.date()

Hopefully they didn't use Yesterday and so on. I didn't find news from yesterday to test it but for the one from today it works. It needs more work if you want to use the exact time from today, not the time of now.

Also I see you have white space in the date/time string. You should also implement the fix number 2 from my suggested fixes in #182.

[AGG2017]

Any feedback? Is it working for you now or you just don't know how to apply the fix?

[wj210]

Hi, i got the same error as well. `results = [] date = None for row in rows: # raw_timestamp = row.xpath("./td")[0].xpath("text()")[0][0:-2] raw_timestamp = str(row.xpath("./td")[0].xpath("text()")[0][0:-2]).strip()

    # if len(raw_timestamp) > 8:
    #     parsed_timestamp = datetime.strptime(raw_timestamp, "%b-%d-%y %I:%M%p")
    #     date = parsed_timestamp.date()
    # else:
    #     parsed_timestamp = datetime.strptime(raw_timestamp, "%I:%M%p").replace(
    #         year=date.year, month=date.month, day=date.day)
    if 'Today' in raw_timestamp:
        parsed_timestamp = datetime.now()
    else:
        parsed_timestamp = datetime.strptime(raw_timestamp, "%b-%d-%y %I:%M%p")
    date = parsed_timestamp.date()

    results.append((
        parsed_timestamp.strftime("%Y-%m-%d %H:%M"),
        row.xpath("./td")[1].cssselect('a[class="tab-link-news"]')[0].xpath("text()")[0],
        row.xpath("./td")[1].cssselect('a[class="tab-link-news"]')[0].get("href"),
        row.xpath("./td")[1].cssselect('div[class="news-link-right"] span')[0].xpath("text()")[0][1:]
    ))

return results`

Is this the correct fix?

[AGG2017]

No, the final result for the function get_news should be:

def get_news(ticker):
    """
    Returns a list of sets containing news headline and url

    :param ticker: stock symbol
    :return: list
    """

    get_page(ticker)
    page_parsed = STOCK_PAGE[ticker]
    news_table = page_parsed.cssselect('table[id="news-table"]')

    if len(news_table) == 0:
        return []

    rows = news_table[0].xpath("./tr[not(@id)]")

    results = []
    date = None
    for row in rows:
        raw_timestamp = str(row.xpath("./td")[0].xpath("text()")[0][0:-2]).strip()

        if len(raw_timestamp) > 8:
            if 'Today' in raw_timestamp:
                parsed_timestamp = datetime.now()
            else:
                parsed_timestamp = datetime.strptime(raw_timestamp, "%b-%d-%y %I:%M%p")
            date = parsed_timestamp.date()
        else:
            parsed_timestamp = datetime.strptime(raw_timestamp, "%I:%M%p").replace(
                year=date.year, month=date.month, day=date.day)

        results.append((
            parsed_timestamp.strftime("%Y-%m-%d %H:%M"),
            row.xpath("./td")[1].cssselect('a[class="tab-link-news"]')[0].xpath("text()")[0],
            row.xpath("./td")[1].cssselect('a[class="tab-link-news"]')[0].get("href"),
            row.xpath("./td")[1].cssselect('div[class="news-link-right"] span')[0].xpath("text()")[0][1:]
        ))

    return results

I just test this function to get the AAPL news (the same example from the front page) and it works. And restart the Python session after applying the fix, otherwise the cached old one will be used if already it is imported.

[AGG2017]

Still no feedback to just say it works, thanks or it doesn't work and to provide the error information. No motivation to waste more of my valuable time on this project when the one asking for help doesn't want to spend even 10 seconds for a simple feedback. You should wait for the official fix releases one day. Sorry.

[scottstables]

Apologies - I can confirm that the fix did indeed work.

[vikyw89]

this is my code:

from finviz import get_news
result = get_news(ticker="AAPL")

data_string = '\r\n            Jan-28-24 08:38PM\r\n      ', format = '%b-%d-%y %I:%M%p'

    def _strptime(data_string, format="%a %b %d %H:%M:%S %Y"):
        """Return a 2-tuple consisting of a time struct and an int containing
        the number of microseconds based on the input string and the
        format string."""
    
        for index, arg in enumerate([data_string, format]):
            if not isinstance(arg, str):
                msg = "strptime() argument {} must be str, not {}"
                raise TypeError(msg.format(index, type(arg)))
    
        global _TimeRE_cache, _regex_cache
        with _cache_lock:
            locale_time = _TimeRE_cache.locale_time
            if (_getlang() != locale_time.lang or
                time.tzname != locale_time.tzname or
                time.daylight != locale_time.daylight):
                _TimeRE_cache = TimeRE()
                _regex_cache.clear()
                locale_time = _TimeRE_cache.locale_time
            if len(_regex_cache) > _CACHE_MAX_SIZE:
                _regex_cache.clear()
            format_regex = _regex_cache.get(format)
            if not format_regex:
                try:
                    format_regex = _TimeRE_cache.compile(format)
                # KeyError raised when a bad format is found; can be specified as
                # \\, in which case it was a stray % but with a space after it
                except KeyError as err:
                    bad_directive = err.args[0]
                    if bad_directive == "\\":
                        bad_directive = "%"
                    del err
                    raise ValueError("'%s' is a bad directive in format '%s'" %
                                        (bad_directive, format)) from None
                # IndexError only occurs when the format string is "%"
                except IndexError:
                    raise ValueError("stray %% in format '%s'" % format) from None
                _regex_cache[format] = format_regex
        found = format_regex.match(data_string)
        if not found:
>           raise ValueError("time data %r does not match format %r" %
                             (data_string, format))
E           ValueError: time data '\r\n            Jan-28-24 08:38PM\r\n      ' does not match format '%b-%d-%y %I:%M%p'

/usr/lib/python3.11/_strptime.py:349: ValueError

[vikyw89]

fix from @AGG2017 works, why didn't it get merged to the master branch ?

[estr95]

I applied the fix from @AGG2017, but am getting a different error message now, so was wondering if something changed more recently.

Using the example: finviz.get_news('AAPL')

The following is returned: ValueError: time data '03:15' does not match format '%I:%M%p'

By chance, does anyone have a new fix for the function get_news to resolve this error ?

Very new to this and not sure what this error message means. Any help would be greatly appreciated!

[mw66]

Simple fix: just add .strip():

$ diff ./finviz/main_func*.py
110c110
<         raw_timestamp = row.xpath("./td")[0].xpath("text()")[0][0:-2]
---
>         raw_timestamp = (row.xpath("./td")[0].xpath("text()")[0][0:-2]).strip()

[estr95]

@mw66 Thank you!