Any didactical example for overfitting?

Hey everyone, I am trying to learn a bit of AI and started coding basic algorithms from scratch, starting wiht the 1957 perceptron. Python of course. Not for my job or any educational achievement, just because I like it.

I am now trying to replicate some overfitting, and I was thinking of creating some basic models (input layer + 2 hidden layers + linear output layer) to make a regression of a sinuisodal function. I build my sinuisodal function and I added some white noise. I tried any combination I could – but I don’t manage to simulate overfitting.

Is it maybe a challenging example? Does anyone have any better example I could work on (only synthetic data, better if it is a regression example)? A link to a book/article/anything you want would be very appreciated.

PS Everything is coded with numpy, and for now I am working with synthetic data – and I am not going to change anytime soon. I tried ReLu and sigmoid for the hidden layers; nothing fancy, just training via backpropagation without literally any particular technique (I just did some tricks for initializing the weights, otherwise the ReLU gets crazy).