Your assumption about connecting red with red to the live (hot) side is correct.
The two black wires on the primary also go together, to the neutral side (although it shouldn't matter whether reds go to live or neutral side).
So, for 120V, the reds go together, and the blacks go together, that is, red with red on one side, black with black on the other side.
If you were connecting them in series for 230V operation, one red wire would go to the hot side, its black counterpart would be shorted with the red one from the other pair, and the black one from that other pair would go to the neutral side of your power.
Your connection schematic/drawing in your question is correct.
To clarify with an explanation from my comment:
Yes, the reds shorted together, and the blacks shorted together, then red ones going to the hot side, and the black ones going to the neutral side.
They are two separate primary coils wound for 120V each, but also for half the total current at 120V.
If you use only one pair of them, they will get warmer/hotter at the full rated load on your secondary because they will have twice their rated current and thus 4 times the amount of heat they were supposed to dissipate, so they could burn out.
Power (wasted as heat) = resistance x the current squared, so double the current means 4 times the heat across the same resistance.