If you need to study some functions but have limited tools you can go a long way assuming everything is linear except at minima/maxima where you assume everything is quadratic.

Use this approach in physics and everything looks like a free field or a perturbation thereof.

Use this in statistics and ML and everything looks like a Gaussian distribution.

So when you see what seems like a deep connection between these domains I wonder if what you're seeing is just the shape of your tools.