Error with tf-keras when trying to geneate random seeds

[fabiancpl]

System Info

I am repeatedly having this error when I try to instantiate the BERT model for pre-training using TensorFlow:

File c:\Users\Fabian Pena\.conda\envs\senlp-tf\Lib\site-packages\tf_keras\src\backend.py:2032, in RandomGenerator._create_seed(self, user_specified_seed)
   [2030](file:///C:/Users/Fabian%20Pena/.conda/envs/senlp-tf/Lib/site-packages/tf_keras/src/backend.py:2030)     return user_specified_seed
   [2031](file:///C:/Users/Fabian%20Pena/.conda/envs/senlp-tf/Lib/site-packages/tf_keras/src/backend.py:2031) elif getattr(_SEED_GENERATOR, "generator", None):
-> [2032](file:///C:/Users/Fabian%20Pena/.conda/envs/senlp-tf/Lib/site-packages/tf_keras/src/backend.py:2032)     return _SEED_GENERATOR.generator.randint(1, 1e9)
   [2033](file:///C:/Users/Fabian%20Pena/.conda/envs/senlp-tf/Lib/site-packages/tf_keras/src/backend.py:2033) else:
   [2034](file:///C:/Users/Fabian%20Pena/.conda/envs/senlp-tf/Lib/site-packages/tf_keras/src/backend.py:2034)     return random.randint(1, int(1e9))

File c:\Users\Fabian Pena\.conda\envs\senlp-tf\Lib\random.py:336, in Random.randint(self, a, b)
    [332](file:///C:/Users/Fabian%20Pena/.conda/envs/senlp-tf/Lib/random.py:332) def randint(self, a, b):
    [333](file:///C:/Users/Fabian%20Pena/.conda/envs/senlp-tf/Lib/random.py:333)     """Return random integer in range [a, b], including both end points.
    [334](file:///C:/Users/Fabian%20Pena/.conda/envs/senlp-tf/Lib/random.py:334)     """
--> [336](file:///C:/Users/Fabian%20Pena/.conda/envs/senlp-tf/Lib/random.py:336)     return self.randrange(a, b+1)

File c:\Users\Fabian Pena\.conda\envs\senlp-tf\Lib\random.py:312, in Random.randrange(self, start, stop, step)
    [309](file:///C:/Users/Fabian%20Pena/.conda/envs/senlp-tf/Lib/random.py:309)     raise ValueError("empty range for randrange()")
    [311](file:///C:/Users/Fabian%20Pena/.conda/envs/senlp-tf/Lib/random.py:311) # Stop argument supplied.
--> [312](file:///C:/Users/Fabian%20Pena/.conda/envs/senlp-tf/Lib/random.py:312) istop = _index(stop)
    [313](file:///C:/Users/Fabian%20Pena/.conda/envs/senlp-tf/Lib/random.py:313) width = istop - istart
    [314](file:///C:/Users/Fabian%20Pena/.conda/envs/senlp-tf/Lib/random.py:314) istep = _index(step)

TypeError: 'float' object cannot be interpreted as an integer

The code to reproduce the error is:

from transformers import (
    BertConfig,
    TFBertForPreTraining,
)

MODEL_CHECKPOINT = "google-bert/bert-base-uncased" 

config = BertConfig.from_pretrained(MODEL_CHECKPOINT)
model = TFBertForPreTraining.from_pretrained(MODEL_CHECKPOINT, config=config)

And these are the dependencies that I am installing:

pip install --upgrade tensorflow==2.16.1 tf-keras==2.16.0 transformers==4.40.2 datasets==2.19.1

After working a lot on this problem I found that it is caused apparently due to compatibility problem with Python. Initially, I was using Python 3.13 (one of the last ones installed with Anaconda), and when I downgrade Python to 3.10.12 I don't get the error.

Who can help?

No response

Information

• [ ] The official example scripts
• [X] My own modified scripts

Tasks

• [ ] An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• [ ] My own task or dataset (give details below)

Reproduction

1. Create the conda environment with the python version of your choice.
2. Install the dependencies.
3. Run the code.

Expected behavior

Run a pre-train job.

[amyeroberts]

cc @Rocketknight1

[Rocketknight1]

Hi @fabiancpl, Python 3.13 was very recently released in beta, and many packages won't work properly with it! I think that's the most likely cause of the issue here, especially since downgrading fixes it. I don't think this is a problem with transformers.