There is a lot more to a nuclear bomb than uranium and plutonium. Containing a volatile nuclear reaction and compressing the uranium to create a nuclear bomb is not an easy task and is something that took the US years to engineer. A country's ability to enrich uranium does not immediately mean they have the ability to make a nuclear weapon. First, extracting uranium from the nuclear power plant is a tedious, costly, and dangerous task, as is the extraction of the correct plutonium from spent nuclear waste. Then, a country must create a bombshell capable of containing the nuclear reaction without detonating, and surround the uranium with enough explosives to compact and explode the nuclear material. Even then, it would take years of refinement for a country to make a powerful nuclear weapon and several rounds of testing that would surely be noticed by the broader global community. Some experts' concerns that nuclear bombs can be readily available from countries that have nuclear power are, by and large, unfounded and gloss over the realities of the materials needed to create such weapons.
While the creation of nuclear weapons is increasingly difficult and costly for a country, a terrorist organization could very easily attempt to make a dirty bomb using spent material from a nuclear waste plant. There is the danger that, if a country has nuclear energy sites, an organization could use the material to craft a poorly engineered yet still effective nuclear weapon. 
Rejecting the premises