Question: If I don't feel passionate about a field should I stick with it for at least my first year, or should I leave before putting my time in?
Background: This fall semester will be my first semester in graduate school. This summer I am doing student research in space physics, which is the field I am pursuing in graduate school. I have also had 2 other summers of research in space physics. Not because I enjoyed the field, but more because they were easy research positions to get at the time.
When I had applied to graduate schools I was desperate to just get in, so I applied to space physics programs only, thinking I had a better shot.
I've heard transferring into a different program in graduate school is really difficult, and you can burn a lot of bridges that way, so I think my only options are to grit my teeth for the next 5-6 years/ try to find some passion in the field/ try reapplying to graduate school from the beginning/ or just get a job.
First, you need to step back and ask yourself why you want to go to graduate school. It's not clear to me that you have a well-defined reason.
An MS+PhD program is not easy for anyone. And it's not just classes that you can grind through. Most of your work will be on doing original research, and that necessarily entails dead ends, setbacks, and overall frustration when things don't work out. Certainly one of the biggest things that keeps successful grad students going is being in love with the subject, so that when things do finally go right it all seems worth it.
There are other motivations too. I've seen everything from making parents proud to challenging oneself to really wanting a degree to boast about.1 Whatever it is, though, you have to ask yourself if, at the end of 5 or 6 (or more!) years of sacrificing some combination of money, social time, and creature comforts, you will like your decision.
On starting over: You'll definitely need to think about the above motivations. Is your heart really set on astrophysics, to the point where you won't have this same doubt even if you get into such a program? This soul searching isn't just to help you make the right decision -- you'll need to explain your thinking to your letter writers and possibly to whatever departments you apply to.
On sticking with it: It's possible you don't have enough information right now. Only you know how much you personally learn to like a subject after studying it intensely, versus how much your first impressions always turn out to be right.2 In this case, you can stick with the program for a year and then decide.3 No one will hold it against you if you drop out. You just have to decide whether that extra information is worth a year of your life.
Be careful, though, if you want to start over with different programs after a year, since the timing of the application season might very well leave you with a gap year. Whatever you do, don't think that you have to be miserable for 5 years.
On getting a job: Ask yourself what kind of career you want, and what you need to do to get there. Are there careers you would enjoy that don't require a degree like this? Remember, any grad school is just a means to an end. There are students who choose grad school in order to delay figuring out what career they really want. Just make sure that's not your motivation.
1Anecdote: I went to an undergrad where the majority of students went on to grad school. This resulted in pressuring a lot of my peers who really didn't have a good motivation to do the same. A large number of them dropped out, and I always wonder how many of them regret the decision to continue schooling in the first place.
2In your particular case, I'm inclined to think your won't find that passion given that you have a nonnegligible amount of experience in the field. Unless you find a project that's considerably different from what you've worked on already and more aligned with your interests (see next footnote), there's not much more information to be had by extra experience. Don't make the mistake of procrastinating on a decision when no new information is headed your way.
3By the way, you are right to recognize that space physics and astrophysics/cosmology are rather disjoint, despite how counterintuitive that is to the uninitiated. That said, there is some overlap, especially on the theoretical side. For example heliophysics is not too distant from stellar structure and astroseismology. Look into what the professors in your department actually research -- you might find something right up your alley.