Yes, you can go back. (Don't ask me for a technical explanation)
In terms of a paradox, consider a paradox is difficult to create as time has it's own (for lack of a better word) 'gravity'. Think of it like a string under tension such as a piano string or tight-rope. If/when you try to (or do) deviate from the set time-line, it is like plucking the string. It creates a type of turbulence and will eventually go back to it's original still and balanced position.
Side effects of doing so include all sorts of metaphysical anomalies effecting the physical world around you, ...and extreme mental instability. Perhaps the effects can be compared to sound waves.
I wonder if there is a way to break it free, to release the tension. Death?
Well you still have an issue there; if you go back in time and change actions (your ripples/sound waves) you ARE editing the past and changing events may lead to an alternative future how ever minor that may be but you would not be the same person that went back as the person that first went back and so the time loop has changed and that's impossible. Also tiny things change drastically depending how far back you go and as we know from the universe everything tenders to work out as intended BUT that's more of an argument that traveling backwards is impossible but traveling forwards is very plausible because we already are doing it; just were talking about on an individual scale
There are several people on this planet who've discovered solid and compelling evidence of time travel to the past, which is in direct conflict with your hypothesis. You're hypothesizing in terms of IF/CAN/CANNOT while others hypothesize HOW.
We don't know how it works yet. What we do know is indicative of it being entirely possible, as per evidence discovered as well as breakthroughs being made in theoretical physics. Thus any position in the matter is somewhat premature. It can only go so far. At this point in time, it is similar to the abstract and obscure nature of a religious debate. ("Substance of things hoped for, evidence of things unseen")
But let's indulge anyway, just for fun.
One can argue that not going back when/how you are 'supposed' to is an equivalent example of deviation. In other words; going back to alter things is equal to not going back if you are destined to do so. Perhaps each time it is done, it sets/resets the form of the time line, and any action not exact to the set form can/will result in turbulence. Maybe that's just part of how it works. ...like an expected effect thereof.