

It does allow you to store your entire codebase. Git LFS (https://git-lfs.com/) also exists if you need to store large files in your repository.
If you mean you want everyone to have identical copies of the code when somebody makes a change, that’s certainly possible, they just have to pull (https://git-scm.com/docs/git-pull) the updates from the remote repository into their local machine.
It’s also possible to automate deployment to production servers. Look into CI/CD if this is something you’re interested in.
As for the original question, I am against the idea of having all developers connect to one machine and do all their work there. Latency is a minor issue but an issue nonetheless. It would also take more effort to track who made what change, while if you had a git repo it would be so easy to review all the modifications that have been made.