Is it ok to do this?

If you accept a job and another (better) job comes along in the time period before you start the new job, is it ok to accept the better job and tell the person who offered you the first job (which you accepted) that you are no longer interested in accepting? Or is that considered unprofessional and it is better to stick with the first job you accept?
This happens all the time when a 'better' offer comes through after someone has accepted a job. Just take the actions you have outlined and you'll be fine. If you want to soften the blow, you could offer to still do PRN at facility #1. Chances are, they won't take you up on the offer, but it will convey the idea that you mean well.
Employers are used to people doing this. Just be professional about the way you go about informing the rejected employer.
It's also a good idea to send a formal letter to the manager thanking them for the opportunity.
Perfectly fine, and they are used to it. Just be professional and thank them for the opportunity in a timely manor...I wouldn't wait until the day before you were supposed to start.