Did you manually pollinate the female flowers? If there isn't sufficient insect activity to help out, you have to do it manually or the fruits will not grow. This is likely what happened as you describe it.
How old is your system? Fruiting plants are demanding on potassium and some other trace elements that newer systems can be lacking. As they mature with time they these will build up in the system water though and support some fruiting plants much better.