many invalid validators still updated metadata

[hwhe]

Describe the bug I actually only have a few hundred validators. In UpdateValidatorMetaDataLoop function, There are thousands of validators in updated validators metadata periodically, This rest api is time-consuming because there are many disused validators.

ssv logs

{"L":"DEBUG","T":"2024-03-07T09:12:52.627Z","N":"Controller","M":"updated validators metadata","validators":3796,"started_validators":0,"took":"16.801082121s"}
{"L":"DEBUG","T":"2024-03-07T09:25:09.087Z","N":"Controller","M":"updated validators metadata","validators":3796,"started_validators":0,"took":"16.363092245s"}
{"L":"DEBUG","T":"2024-03-07T09:37:26.888Z","N":"Controller","M":"updated validators metadata","validators":3796,"started_validators":0,"took":"17.706500315s"}
{"L":"DEBUG","T":"2024-03-07T09:49:42.898Z","N":"Controller","M":"updated validators metadata","validators":3796,"started_validators":0,"took":"15.910319136s"}
{"L":"DEBUG","T":"2024-03-07T10:01:58.957Z","N":"Controller","M":"updated validators metadata","validators":3796,"started_validators":0,"took":"15.964554241s"}
{"L":"DEBUG","T":"2024-03-07T10:14:18.252Z","N":"Controller","M":"updated validators metadata","validators":3796,"started_validators":0,"took":"19.180227538s"}

It shows here that there are 3796 validators, Actually, I only have 300. and the interface is very time consuming. Waste of cpu resources on my node

// UpdateValidatorMetaDataLoop updates metadata of validators in an interval
func (c *controller) UpdateValidatorMetaDataLoop() {
	var interval = c.beacon.GetBeaconNetwork().SlotDurationSec() * 2

	// Prepare share filters.
	filters := []registrystorage.SharesFilter{}

	// Filter for validators who are not liquidated.
	filters = append(filters, registrystorage.ByNotLiquidated())

	// Filter for validators which haven't been updated recently.
	filters = append(filters, func(s *ssvtypes.SSVShare) bool {
		last, ok := c.metadataLastUpdated[string(s.ValidatorPubKey)]
		return !ok || time.Since(last) > c.metadataUpdateInterval
	})

	for {
		time.Sleep(interval)
		start := time.Now()

		// Get the shares to fetch metadata for.
		shares := c.sharesStorage.List(nil, filters...)
		var pks [][]byte
		for _, share := range shares {
			pks = append(pks, share.ValidatorPubKey)
			c.metadataLastUpdated[string(share.ValidatorPubKey)] = time.Now()
		}

		// TODO: continue if there is nothing to update.

		c.recentlyStartedValidators = 0
		if len(pks) > 0 {
			err := beaconprotocol.UpdateValidatorsMetadata(c.logger, pks, c, c.beacon, c.onMetadataUpdated)
			if err != nil {
				c.logger.Warn("failed to update validators metadata", zap.Error(err))
				continue
			}
		}
		c.logger.Debug("updated validators metadata",
			zap.Int("validators", len(shares)),
			zap.Uint64("started_validators", c.recentlyStartedValidators),
			fields.Took(time.Since(start)))

		// Notify DutyScheduler of new validators.
		if c.recentlyStartedValidators > 0 {
			select {
			case c.indicesChange <- struct{}{}:
			case <-time.After(interval):
				c.logger.Warn("timed out while notifying DutyScheduler of new validators")
			}
		}
	}
}

the validators info got from sharesStorage, The Expired validators are not cleared in time.

I think it can be optimized here.

To Reproduce It's an inevitable question.

Expected behavior The metadata of invalid validatos does not need to be obtained.

Screenshots If applicable, add screenshots to help explain your problem.

Node (please complete the following information):

• Operator Version: [ v1.3.3]
• BN Client [lighthouse, v5.1.3]

Additional context Add any other context about the problem here.