Yes politically unlikely.It definitely isn't politically or economically feasible for Australia to start a nuclear industry now, especially for the small fleet it would ever own
However, the idea isn’t (necessarily) to start a full blown nuclear industry. It would be sufficient to be able to operate nuclear subs with extensive external assistance (including building the reactors, enriching and disposing the fuel elsewhere). That should be an order of magnitude easier.
The sub fleet to be supported would be larger than many nuke sub operators (UK, France, Brazil, India) so there might be enough scale to make it economically feasible.
Again, not saying it’s easy. Just the current path doesn’t strike me as being particularly easy (or cheap) either.