Asking for domain adaptation method suggestion

[offchan42]

Suppose I train a ML model to classify human face (from direct frontal angle in a day environment) as either male or female on a large labeled dataset. Let call the model X. Then later I collect more data with different camera angle, e.g. the camera is now looking at the face with 45 degrees in a night environment, I labeled a few samples for this new dataset. If I want to do transfer learning from model X to the new dataset, what's the best approach? My goal is to predict new dataset accurately with few labeled samples. Because the cost of labeling is high in my problem. I don't care about the accuracy of the first dataset. I only care about the accuracy of the new dataset.

E.g. I know that you could just re-train only the last layer from the model X and freeze early layers for the new dataset, but maybe that approach is for datasets with a similar distribution like ImageNet images and cats/dogs images? ImageNet models surely have seen many kinds of real-world objects so it's able to classify cats and dogs when transfer learning.

But for my case, both datasets differ slightly in distribution/domain, that is the day/night environment and angle. And the model has seen only the frontal angle for its entire life. How can it adapt to a 45-degree angle the best?

If you know any technique/tutorial/paper that might work, let me know. It would be best if you have experience with it.