Organic (or near organic) fish

[Carl Story]

I am totally new to this, but have been wanting to try it for a long time.   I'm recently retired, and establishing a simple home system is on my retiree list.   I've read the book (Aq Gardening) and THINK I'm about ready to build a simple, 2 barrel 1:1 type home system.  I want to start small (size & investment wise) while learning, with a goal of going bigger once I have some experience. 

I do not have consumption of the fish as a top priority, but rather the growing of organic greens & herbs.  My wife is all excited about this if (capital IF) I can keep it organic.  

Now...for the fish.  I've read online all the arguments about what constitutes organic fish, the USDA position, etc.  As I don't plan to consume the fish, I'd like to go with goldfish due to the lower cost.  If I buy "normal" goldfish (think PetSmart) and feed them only organic feed, then would my plants be 'organic'?  If not, then how do I get started with the fish?

Thank you for your learned advice.