How high did the PH get? Did you try to adjust too fast? Trying to change it when the gravel is buffering it can cause PH swings that could have stressed your fish. A high PH will probably affect plants more than fish in most cases.
I have read that PH adjustments should not be more than .1 per day. One full point is 100 times less or more acidic/alkaline. If your gravel is buffering that strongly then putting in the PH down probably just bounced PH down, then it shot back to where it was.
Are you gonna change out your gravel and try again? So sorry to hear about your fish.
Hard to say if all of that is going to be the gravel or the water outgassing changing your pH.
If the gravel is limestone, soaking it in acid won't leave you much.
I hate to be a bearer of bad news but choosing gravel that is a quarter of an inch in size and smaller - all the way to fine sand grains is just asking for trouble. It looks nice at first. You think it should be ideal for Aquaponics but wait 6 to 12 months and you will have problems with your system.
The size of your gravel is just too small. You will slowly create anaerobic black zones where fresh oxygen will not be able to penetrate to oxygenate plant roots or microscopic bacteria. In fact anaerobic bacteria will colonize your system and other negative issues will arise.
Your dilemma reminds me of Murray Hallam's Aquaponics Secrets DVD trailer where Murray succinctly explains the very problem that you have shared with us all and how many people see his system and then reject his advice and do exactly what you have done. Choose gravel that they think plants will like....
See this link for a hint and an explanation of your problem:
You really need to get rid of your pea gravel.
Toss it out completely and get 20mm gravel or small pebbles about an inch in diameter.
I know this sound extraordinary but you need to create air pockets in your gravel grow bed system for oxygen to penetrate.
For worms to wriggle through...
For plant roots to penetrate and thrive...
Spaces for air pockets is crucial.
Hard to believe but this system that Murray has created will enable you to grow plants for over 4 years without cleaning the grow bed system and to it enable you to enjoy your Aquaponics journey with a trouble free system.
I guess I hadn't paid very close attention to size on the gravel. My gravel is 1/2" brown river rock, which means it fit through the 1/2" mesh screen when they separate the gravel into different products. So my gravel is 1/2" and smaller. It seems like fine media to me but I have a good population of composting worms. I think a mix between 1/2" and 3/4" would be even better but I've never found that size mix in an affordable option that wasn't limestone.
I personally think 1/4" and smaller may well give you some issues long term.