
\documentclass[10pt]{article}
\usepackage{multicol}
\usepackage{graphicx}
\usepackage{wrapfig}
\usepackage{subfigure}
\usepackage{setspace}
\usepackage{scrextend}
\usepackage{hyperref}
\usepackage[left=.5in,right=.5in,bottom=1in,top=.5in]{geometry}

\bibliographystyle{IEEEtran}

\begin{document}
\doublespacing


\begin{center}
\section*{The Hub Test Module (HTM)}
Version 0.9, 12/25/2017\\
\end{center}
\section{Introduction}
The purpose of this document is to describe the requirements and proposed implementation for the Hub Test Module (HTM). 

\section{Requirements}
\subsection{Example}
Requirements for the HTM are specified in the following format:

\begin{labeling}{HTM-999}
	\item [HTM-\#\#\#] \textbf{Brief description of the requirement}

	Detailed description of the requirement if necessary, optional 'nice to haves' that are related to the minimum requirement will be described here.
\end{labeling}

\subsection{HTM Requirements}
\begin{labeling}{HTM-XXX}

	\item [HTM-000] \textbf{The HTM shall fit in an ATCA slot.}
	
	The HTM must fit in the envelope of one ATCA slot, as described in the ATCA specification.  A full size board is preferred, such that it is easy to insert and remove HTMs in a fully populated shelf (2 Hubs, 12 HTMs)
	
	\item [HTM-005] \textbf{The HTM shall contain an FPGA of sufficient capability to fulfill all of its requirements}
	
	General requirement indicating that we want the flexibility of an FPGA.  Xilinx is preferred, so that we can hopefully leverage work from the Hub and FTM firmware. 

	\item [HTM-010] \textbf{The HTM shall be powered from the ATCA backplane -48V power infrastructure}
	
	No ATCA Power Entry Module, no Redundant Input Power, not Hot-Swappable.  A or B power can be selected by populating the relevant fuses on the HTM.

	\item [HTM-015] \textbf{The HTM shall not require external connections during production testing.}
	
	No external connections allowed during production testing.  Ethernet and JTAG connectors will be present on the front panel.  This requirement does not prevent the ethernet to ethernet connections on the front panel of the HTM.

	\item [HTM-020] \textbf{The HTM shall configure itself upon power-up.}

	The firmware must be loadable onto the HTM itself, and it must boot into that firmware automatically upon power-up.

	\item [HTM-025] \textbf{The HTM shall provide 6 Multi-Gigabit Transceiver (MGT) channels of FEX-like data to each Hub via the ATCA backplane}
	
	\item [HTM-030] \textbf{The HTM shall receive 1 MGT channel of combined data from Hub1}
	
	Optionally, the HTM could receive 1 MGT channel of combined data from Hub2 as well.
	
	\item [HTM-035] \textbf{The HTM shall be capable of receiving and validating the combined data}
	
	\item [HTM-040] \textbf{The HTM shall receive and lock to the 40.08 MHz LHC clock from Hub1, via the ATCA backplane.}

	40.08 MHz clock from Hub2 to be routed to a pair of SelectIO pins, merely to check that it exists.

  \item [HTM-045] \textbf{The HTM shall generate a 320.64 MHz clock locked to the 40.08 MHz LHC clock from Hub1.}
	
	\item [HTM-050] \textbf{The HTM shall use the generated 320.64 MHz clock as a reference for its MGTs.}

	\item [HTM-055] \textbf{The HRM shall provide the 40.08 MHz and 320.64 MHz clocks to its FPGA for use as logic clocks.}
	
	\item [HTM-060] \textbf{The HTM MGTs shall operate at 6.4 Gbps minimum.}
	
	Optionally, the HTM MGTs can operated at 9.8 to 10.2 Gbps.
	
	\item [HTM-065] \textbf{The HTM shall communicate with Hub1 via Ethernet over the ATCA backplane.}
	
	Communicating to Hub2 via Ethernet over the ATCA backplane is desirable as well, but not a hard requirement. 
	
	\item [HTM-070] \textbf{The HTM shall detect which ATCA slot it is in via the backplane hardware slot ID pins.}
	
	\item [HTM-075] \textbf{The HTM shall generate a MAC address from the hardware slot ID}
	
	\item [HTM-080] \textbf{The HTM shall generate a static IP address from the hardware slot ID}
	
	\item [HTM-085] \textbf{The HTM shall provide several status LEDs on the front panel.}

	Specific LEDs TBD, ideas: standby and main power, main ethernet LED, and a small number of general Select IO LEDs.

\end{labeling}								


\section{Hardware Description}

It is proposed that the HTM consist of the following:

\begin {enumerate}
	\item One Trenz Electronic TE0782 XC7Z035-2FFG900I module, hereafter referred to as the 'FPGA module' or the 'TE0782'.
	\item One carrier board with sufficient circuitry and mechanical hardware to support that module, hereafter referred to as the 'HTM carrier board'
\end {enumerate}

\subsection {Hardware Details - Mechanical}

\subsubsection{Envelope}

The TE0782 is 8.5 cm square, has a PCB thickness of 1.7mm, and a mating height of 5mm when using standard connectors.  This is well within the envelope of an ATCA slot.

\subsubsection{Connectors}

The TE0782 mounts to the carrier board via three high speed Samtec ASP-122952-01 Board-to-Board (B2B) connectors.  HTM connector naming has been chosen to be consistent with the TEBT0782 base board.  This makes the HTM consistent with the Trenz Electronic master pinout table, which is referenced above.  Table \ref{tab:conn} shows which TE0782 connectors mate to the three B2B connectors on the HTM.

\begin{table}[!htb]
\caption{HTM and TE0782 Connector Mating}\label{tab:conn} 
\begin{center}
 \begin{tabular}{|c | c |}
 \hline 
 HTM Connector & TE0782 Connector \\
 \hline \hline
 J1B & J2 \\ \hline
 J2B & J1 \\ \hline
 J3B & J3 \\ \hline
 \end{tabular}
\end{center}
\end{table}

\begin{figure}[htp]
  \begin{center}
    \includegraphics[width=4in]{te0782_phys.png}
  \end{center}
  \caption{TE0782 Bottom View}
  \label{fig:te0782_bottom}
\end{figure}

Figure \ref{fig:te0782_bottom} shows the locations of the B2B connectors on the bottom side of the TE0782 module.

\begin{figure}[htp]
  \begin{center}
    \includegraphics[width=4in]{physical.png}
  \end{center}
  \caption{HTM TE0782 Module Orientation}
  \label{fig:physical}
\end{figure}

Figure \ref{fig:physical} shows a notional top view of the TE0782 module on the HTM carrier card.    


The SelectIO lines on TE0782 J1 are 1.8V tolerant, while those on TE0782 J2 and TE0782 J3 are 3.3V tolerant.

JTAG will be exposed on the front panel, in case we need to load new firmware in-situ.

\subsubsection{PCB Stackup}

One big question surrounding the layer requirements for power purposes is the power consumption on the 12V rail.  Currently, the TE0782 documentation lists "`TBD"' on this point.  One approach could be to look up the maximum input current per pin on the B2B connectors, and multiply this by the number of 12V input pins.  There is also a power estimator spreadsheet from Xilinx for the XC7Z035. My gut feeling is that two 1oz planes, or possibly even 0.5oz planes would be sufficient given that the module accepts 12V main power.

For routing the high speed traces, it is expected that four layers will be sufficient (four layers are required on the Hub board to escape the zone 2 connector).  The board stackup looks like:


\singlespacing
high speed diff pairs 1 (TOP)

GND

high speed diff pairs 2 

GND

Power 1

GND

Power 2

GND

high speed diff pairs 3

GND

high speed diff pairs 4 (BOTTOM)


\doublespacing

\subsection {Hardware Details - Electrical}

TE0782 schematics are available here: \url{https://www.trenz-electronic.de/fileadmin/docs/Trenz_Electronic/TE0782/REV02/Documents/SCH-TE0782-02-035-2I.PDF}

The pinout table for the B2B connectors is available here: \url{http://www.trenz-electronic.de/fileadmin/docs/Trenz_Electronic/Pinout/TE_MASTER_PINOUT.xlsm}

\subsubsection{Power}

The TE0782 requires two input voltages, 3.3V Standby power for the on-board CPLD, and 12V Main Power for the rest of the board.  The HTM carrier board must create these voltages from the -48V input power provided by the ATCA crate.  It is proposed that -48V to 12V conversion use the same +12V isolated power module as the Hub design.  The TE0782 draws 100 mA from the 3.3V rail.  Thus, the LDO will dissipate: (V\textsubscript{i} - V\textsubscript{o})*I = (12v - 3.3V) * 0.1A = 0.87W.

Note that in the connector pinout and the schematics, C3.3V is the input power to the CPLD, 3.3V is the output of one of the onboard power supplies (U16, LTM4644EY\#PBF).

The 3.3V supply is provided by an LM1117T-3.3 LDO linear regulator.

The B2B connector pins for power are shown in Table \ref{tab:power}

\begin{table}[!htb]
\caption{TE0782 Power Pins}\label{tab:power} 
\begin{center}
 \begin{tabular}{|c | c |}
 \hline 
 Power Rail & Pins \\
 \hline \hline
 3.3V STBY  &  J2-147, J2-148 \\
 \hline
 12V Main & J2-165, J2-166, J2-167, J2-168  \\
 \hline
 \end{tabular}
\end{center}
\end{table}

Grounds are distributed through J1, J2, and J3.

\subsubsection{Clocks}

The TE0782 has an onboard Si5338 PLL for GTX clocking.  This PLL can accept an input clock, and can be programmed via I2C.  Currently the plan is to receive the clock from Hub1.  There are no plans to clock the TE0782 from the Hub2 clock.

The onboard PLL will multiply the  40.08 MHz LHC reference clock to 320.64 MHz.  Figure \ref{fig:clks} shows this design.

\begin{figure}[htp]
  \begin{center}
    \includegraphics[width=4in]{clks.png}
  \end{center}
  \caption{HTM Clock Design}
  \label{fig:clks}
\end{figure}


In either case, either one or two differential clocks must be connected to the B2B connectors shown in Table \ref{tab:clocks}.


\begin{table}[!htb]
\begin{center}
 \caption{TE0782 Clock Inputs}\label{tab:clocks} 
 \begin{tabular}{|c | c | c|}
 \hline 
 Clock & Positive Pin & Negative Pin \\
 \hline \hline
 Si5338 Input & J3-38 & J3-40 \\
 \hline
 GT REFCLK1 & J3-37 & J3-39 \\
 \hline
 GT REFCLK2 & J1-38 & J1-40 \\
 \hline
 \end{tabular}

Since we are using the Si5338 PLL, we will connect the 40.08 MHz LHC clock from Hub 1 to its input.

Prior to the carrier board arriving, the 25 MHz clock on the TE0782 module can be used to test basic functionality.
 
\end{center}
\end{table}

\subsubsection{MGTs}

The TE0782 has 16 GTX Transceivers exposed on the high speed Samtec B2B connectors.  These are capable of up to 12.5 Gbps.

The relevant pins on the B2B connectors for the MGTs are shown in Table \ref{tab:mgts} 

\begin{table}[!htb]
 \caption{TE0782 MGT pinouts}\label{tab:mgts} 
\begin{center}
 \begin{tabular}{|c | c | c | c | c | c |}
 \hline 
MGT \# & Role & RxP & RxN & TxP & TxN \\
 \hline \hline
 0 & Hub 1 Tx0 & J3-32 & J3-30 & J3-31 & J3-29\\ \hline
 1 & Hub 1 Tx1 & J3-28 & J3-26 & J3-27 & J3-25\\ \hline
 2 & Hub 1 Tx2 & J3-24 & J3-22 & J3-23 & J3-21 \\ \hline
 3 & Hub 1 Tx3 & J3-20 & J3-18 & J3-19 & J3-17 \\ \hline
 4 & Hub 1 Tx4 & J3-16 & J3-14 & J3-15 & J3-13 \\ \hline
 5 & Hub 1 Tx5 & J3-12 & J3-10 & J3-11 & J3-9 \\ \hline
 6 & Hub 1 Combined Data & J3-8 & J3-6 & J3-7 & J3-5 \\ \hline
 7 & Not used & J3-4 & J3-2 & J3-3 & J3-1 \\ \hline \hline
 MGT \# & Role & RxP & RxN & TxP & TxN \\ \hline \hline
 8 & Hub 2 Tx0 & J1-1 & J1-3 & J1-2 & J1-4 \\ \hline
 9 & Hub 2 Tx1 & J1-5 & J1-7 & J1-6 & J1-8 \\ \hline
 10 & Hub 2 Tx2 & J1-9 & J1-11 & J1-10 & J1-12\\ \hline
 11 & Hub 2 Tx3 & J1-13 & J1-15 & J1-14 & J1-16 \\ \hline
 12 & Hub 2 Tx4 & J1-17 & J1-19 & J1-18 & J1-20 \\ \hline
 13 & Hub 2 Tx5 & 1-21 & J1-23 & J1-22 & J1-24 \\ \hline
 14 & Hub 2 Combined Data & J1-25 & J1-27 & J1-26 & J1-28 \\ \hline
 15 & Not used & J1-29 & J1-31 & J1-30 & J1-32 \\ \hline
 \end{tabular}
\end{center}
\end{table}

The TE0782 MGTs are connected to backplane connector J23 via the carrier board as shown in Table \ref{tab:highspeed}

\begin{table}[!htb]
 \caption{TE0782 MGTs to Backplane}\label{tab:highspeed} 
\begin{center}
 \begin{tabular}{|c | c | c | c | c | c |}
 \hline 
MGT \# & Role & B2B-P pin & B2B-N pin & Backplane-P pin & Backplane-N pin\\
 \hline \hline
 0 & Hub 1 Tx0 & J3-31 & J3-29 & J23-A4 & J23-B4\\ \hline
 1 & Hub 1 Tx1 & J3-27 & J3-25 & J23-E4 & J23-F4 \\ \hline
 2 & Hub 1 Tx2 & J3-23 & J3-21 & J23-A3 & J23-B3 \\ \hline
 3 & Hub 1 Tx3 & J3-19 & J3-17 & J23-E3 & J23-F3 \\ \hline
 4 & Hub 1 Tx4 & J3-15 & J3-13 & J23-C3 & J23-D3 \\ \hline
 5 & Hub 1 Tx5 & J3-11 & J3-9 & J23-G3 & J23-H3 \\ \hline
 6 & Hub 1 Combined Data & J3-8 & J3-6 & J23-G4 & J23-H4  \\ \hline\hline
 MGT \# & Role & B2B-P pin & B2B-N pin & Backplane-P pin & Backplane-N pin \\ \hline \hline
 8 & Hub 2 Tx0 & J1-2 & J1-4 & J23-A2 & J23-B2 \\ \hline
 9 & Hub 2 Tx1 & J1-6 & J1-8 & J23-E2 & J23-F2 \\ \hline
 10 & Hub 2 Tx2 & J1-10 & J1-12 & J23-A1 & J23-B1\\ \hline
 11 & Hub 2 Tx3 & J1-14 & J1-16 & J23-E1 & J23-F1 \\ \hline
 12 & Hub 2 Tx4 & J1-18 & J1-20 & J23-C1 & J23-D1 \\ \hline
 13 & Hub 2 Tx5 & J1-22 & J1-24 & J23-G1 & J23-H1 \\ \hline
 14 & Hub 2 Combined Data & J1-25 & J1-27 & J23-G2 & J23-H2  \\ \hline \hline
 \end{tabular}
\end{center}
\end{table}



\subsubsection{Ethernet}

The TE0782 has two onboard Gigabit Ethernet Transceiver PHYs, though one is dedicated to the Cortex-A9 ARM MPCore.  The other PHY is available to the FPGA directly, and will be sufficient to communicate to Hub1 via the backplane connector J23.

The TE0782 also has two onboard Ethernet MAC Address EEPROMs, one for each GigE channel.

Magnetics will be required for each Ethernet interface.  The HTM carrier board will use the same Pulse Engineering HX5201NL magnetics as the Hub does.  

Both Ethernet interface from the TE0782 will be routed to TE Connectivity 1-406541-1 RJ-45 connectors on the front panel of the HTM, via the HX5201NL magnetics.  Both Ethernet interfaces from the backplane will routed directly to TE Connectivity 1-406541-1 RJ-45 connectors on the front panel of the HTM.  This is shown in Figure \ref{fig:eth}.


\begin{figure}[htp]
  \begin{center}
    \includegraphics[width=4in]{ethernet.png}
  \end{center}
  \caption{HTM Ethernet Lines}
  \label{fig:eth}
\end{figure}

The Ethernet pinout at the TE Connectivity 1-406541-1 RJ-45 connectors will be as shown in Table \ref{tab:rj45}:


\begin{table}[!htb]
 \caption{TE0782 RJ-45 connector pinouts}\label{tab:rj45} 
\begin{center}
 \begin{tabular}{| c | c |}
 \hline 
RJ-45 Pin & Signal \\ \hline \hline
1 & A+\\ \hline
2 & A-\\ \hline
3 & B+\\ \hline
4 & C+\\ \hline
5 & C-\\ \hline
6 & B-\\ \hline
7 & D+\\ \hline
8 & D-\\ \hline
9 & Analog GND\\ \hline
10 & Analog GND\\ \hline
 \end{tabular}
\end{center}
\end{table}

The LED signals for each Ethernet interface are routed to the FPGA.  SelectIO lines will have to be used to illuminate LEDs on the carrier board for these signals.

The B2B connections for the Ethernet PHYs are described in Table \ref{tab:eth}

\begin{table}[!htb]
 \caption{TE0782 Ethernet pinouts}\label{tab:eth} 
\begin{center}
 \begin{tabular}{| c | c | c | c |}
 \hline 
PHY1 Signal & Pin & PHY2 Signal & Pin \\ \hline \hline
PHY1\_MDI0\_P & J2-23 & PHY2\_MDI0\_P & J2-39 \\ \hline
PHY1\_MDI0\_N & J2-21 & PHY2\_MDI0\_N & J2-37 \\ \hline
PHY1\_MDI1\_P & J2-19 & PHY2\_MDI1\_P & J2-35 \\ \hline
PHY1\_MDI1\_N & J2-17 & PHY2\_MDI1\_N & J2-33 \\ \hline
PHY1\_MDI2\_P & J2-15 & PHY2\_MDI2\_P & J2-31 \\ \hline
PHY1\_MDI2\_N & J2-13 & PHY2\_MDI2\_N & J2-29 \\ \hline
PHY1\_MDI3\_P & J2-11 & PHY2\_MDI3\_P & J2-27 \\ \hline
PHY1\_MDI3\_N & J2-9 & PHY2\_MDI3\_N & J2-25 \\ \hline
 \end{tabular}
\end{center}
\end{table}

The L1Calo backplane specification defines Base Channels 1 and 2 are used for communicating to Hub 1 and Hub 2 respectively.  These channels use the pins shown in \ref{tab:bpeth}

\begin{table}[!htb]
 \caption{Backplane Ethernet Pinouts}\label{tab:bpeth} 
\begin{center}
 \begin{tabular}{| c | c | c | c |}
 \hline 
Base Channel & Lane & Positive Pin & Negative Pin \\ \hline \hline
1 & 0 & J23-5A & J23-5B \\ \hline
1 & 1 & J23-5C & J23-5D \\ \hline
1 & 2 & J23-5E & J23-5F \\ \hline
1 & 3 & J23-5G & J23-5H \\ \hline
2 & 0 & J23-6A & J23-6B \\ \hline
2 & 1 & J23-6C & J23-6D \\ \hline
2 & 2 & J23-6E & J23-6F \\ \hline
2 & 3 & J23-6G & J23-6H \\ \hline
 \end{tabular}
\end{center}
\end{table}


\subsubsection{Hardware ID}

Hardware ID pins from the backplane will be routed to selectIO lines via J2.  These pins are tied to 3.3V via a 9.1kΩ resistor, and to analog ground via a 1nF bypass capacitor.  See requirements 2.298 and 2.299 in the ATCA base specification.  
	
	NOTE: The Hub uses 1kΩ resistors and 100 nF capacitors for these lines.


\section{Firmware Description}

\subsection{Configuration}

The TE0782 has a 32 MB SPI flash available for configuration and/or bootloader storage.  The First Stage Bootloader must exist on the SPI flash, and there is 4 GB of eMMC available for secondary boot.

\subsection{Ethernet Control}

TBD - expect to use similar IPbus infrastructure as the Hub, receiving commands from the backplane ethernet connection to Hub1. Also include dummy registers for Hub random register test through switch.

\subsection{FEX Data Generation}

TBD - Aurora-capable, also fixed lanes to emulate jFEX?  Defined short and long duration patterns, pseudo-random patterns, eventually real-looking data.

\subsection{Hub Combined Data Processing}

TBD



\end{document}