Hubs 17, 28, and 30 have had repeated power supply issues. Supplies would fail after testing good, or after
running for a period of time on the bench and even in the shelf. This has been traced to one of the fluxes
I've been using during final assembly.

Specifically, ChipQuik CQ2LF-16. This is billed as a lead-free, water-soluble flux; it is very 
thin (water-like) and is blue in color. 

This flux is electrically conductive, and unless it's been 100% removed from components it's been used on,
it essentially becomes a resistor in parallel with whatever it's under. 

The problem lies in that I've been using isopropyl alcohol to clean the boards as I worked, and not water. 
So when a little bit of residue happens to be left underneath, say a capacitor, DC will be conducted where
it was supposed to be blocked.

This led to a number of issues with Hubs 17, 28, and 30. 

I'm using high-pressure G3 flux remover to go back and clean all the hubs. I'll go back and make a note in
each trailer sheet to confirm that it's been cleaned.

---------------------
UPDATE: SEP 08, 2020
---------------------

Hub SN14 (at CERN since Feb 2020) ran for a period of time in STF and has since failed because of this issue.

Investigation into the cleaning method spoken of above has turned up that not all the residue is being
removed from underneath affected components. 

After an email exchange with Sylvain Kaufman, I pulled SN13 from the storage rack. SN13 had 
been through the above cleaning process, and in lifting CXX51/CXX01 in each of the seven supplies, I found 
residue still smeared underneath them. Further, I started lifting random resistors that I had placed, and was 
finding residue under them, too.

After sharing these findings with Wade, I/we decided that the only practical solution to this is to remove
and replace every cap and resistor on the board using regular rosin flux after cleaning the affected area. 
My focus is mainly in the power supplies, and in the Hub power control section.

- 0 Ohm jumpers shouldn't be affected by this, and are being excluded from this process.
- The PLLS are a somewhat special case, as they all have a layer of Kapton tape on the bottom. For now, I'm
leaving those alone, as they can't be removed from the board without a significant chance of destroying them,
and we only have a handul of each left.

I'll be making a note in each Hub's trailer to confirm this process has been completed, starting with SN20
and continuing through SN30, then going back to 13-19.

Hub SN14 has been decommissioned and Yuri will eventually have to send it back to undergo this process.




James