Hub-Module Base Interface Ethernet Switch ----------------------------------------------- Original Rev. 26-Feb-2015 Current Rev. 30-Mar-2016 The Hub Module contains a Base Interface Ethernet Switch that is built from 3 Broadcom BCM 53128 KQLE switch chips. The Hub Module Ethernet switch provides ATCA Base Interface Ethernet connections to the following: 12 to the FEX Cards via the backplane 1 to the Other Hub's FPGA via the backplane 1 to This Hub's FPGA via capacitor coupling 4 Front Panel RJ-45s 1x to Switch Chip "A", 2x to Switch Chip "B", 1x to Switch Chip "C" The general layout of the Hub's Base Interface Ethernet Switch and it connections in the L1Calo ATCA Shelf are shown in drawings #3 and #4. During normal operation the 4 front panel RJ-45 connections to the Ethernet Switch are used in the following way: - one is used for the "up-link" connection, - one is used for the connection to the ROD or IPMC mezzanine on This Hub Module. - one is used for the connection to the ROD or IPMC mezzanine on the Other Hub Module. and there is one spare front panel RJ-45 connection to the Ethernet Switch. Internally there are 4 switch ports used to connect the 3 BMC53128 switch chips: Chip "A" <--> Chip "B" and Chip "B" <--> Chip "C". The Hub Module does not provide a Base Interface Ethernet connection to the Shelf Manager. With 2 Hub Modules in an ATCA Shelf the intent is that the switch in one module will provide connection to the "Control" Ethernet and the switch in the other Hub Module will proved connection to the "DCS" Ethernet. The FEX modules have an ethernet connection to both switches and thus to both networks. The Hub FPGAs also have an etherent connection to both switches and thus to both networks. The RODs have only one ethernet connection and both the ROD on This Hub and the ROD on the Other Hub are connected to the Control network switch. The IPMCs have only one ethernet connection and both the IPMC on This Hub and the IPMC on the Other Hub are connected to the DCS network switch. Test systems may be operated with a single Hub Module that handles the Etherent connects to both the single ROD and single Hub IPMC in the Shelf. The Hub's ethernet switch supports 10/100/1000 Base-T operation. The switch is based on 3 Broadcom BCM53128 switch chips. These 3 chips are connected together to form one switch but can also be operated as 2 separate switches. Note that the part used is a BCM53128 not a BCM53128v. Power Supply for the BCM53128 Switch Chip: ------------------------------------------ The BCM53128 Switch chip has some rather specific power supply requirements in terms of the number of supplies, their noise level, and the order in which they are ramped up at power on. The naming of the power buses to this switch chip does not seem to be consistent in all of the BCM53128 documentation so one must be very careful. In the Hub design I have used unambiguous versions of the BMC53128 power supply net names. Using the power supply net-names from the Data Sheet the required power rails are: Data Sheet Power Supplies Net Power to Details ----- ---------------- ---------------------------------- AVDDH Analog I/O 3.3V from BULK_3V3 Net-name matches in the Design Guide, the Demo Brd Sch, and Data Sheet tables and Pin Lists. AVDDL Analog Core 1.2V from SWCH_1V2 DCDC Conv. Ferrite isolated on Demo Board. Net-name matches in the Design Guide, the Demo Brd Sch, and Data Sheet tables and Pin Lists. DVDD Digital Core 1.2V from SWCH_1V2 DCDC Conv. Called DVDD in Data Sheet and Sch. Called VDDC in the Design Guide. OVDD GMII, RGMII Port can be: 1.5V or 2.5V or 3.3V from BULK_3V3 on the Hub. This port is not used on the Hub. Called OVDD in Data Sheet and Sch. Called VDDO1 in the Design Guide. OVDD2 Digital 3V3 I/O 3.3V from BULK_3V3 Called OVDD2 in Data Sheet and Sch. Called VDDO3 in the Design Guide. GPHY1_BAVDD Phys Analog 3.3V from BULK_3V3 Ferrite isolated on Demo Brd. Called GPHY1_BAVDD in Data Sheet Table 32 Signal Definitions, and Design Guide Pg 62. Called GPHY1_BVDD in the Schematic, and the Data Sheet Pin Lists. Called QGPHY1_BVDD1 in the Design Guide Pg 61. GPHY2_BAVDD Phys Analog 3.3V from BULK_3V3 Ferrite isolated on Demo Brd. Called GPHY2_BAVDD in Data Sheet Table 32 Signal Definitions, and Design Guide Pg 62. Called GPHY2_BVDD in the Schematic, and the Data Sheet Pin Lists. Called QGPHY2_BVDD1 in the Design Guide Pg 61. PLL_AVDD PLL Analog 1.2V from SWCH_1V2 DCDC Conv. Ferrite isolated on Demo Brd. Called PLL_AVDD in all documents but not listed Pg 61 Design Guide. GPHY1_PLLDVDD Phys Analog 1.2V from SWCH_1V2 DCDC Conv. Ferrite isolated on Demo Brd. Called GPHY1_PLLDVDD in Data Sheet Table 32 Signal Definitions, and Design Guide Pg 62. Called GPHY1_PLLVDD in the Schematic, and the Data Sheet Pin Lists. Called QGPHY1_PLLVDD in the Design Guide Pg 61. GPHY2_PLLDVDD Phys Analog 1.2V from SWCH_1V2 DCDC Conv. Ferrite isolated on Demo Brd. Called GPHY2_PLLDVDD in Data Sheet Table 32 Signal Definitions, and Design Guide Pg 62. Called GPHY2_PLLVDD in the Schematic, and the Data Sheet Pin Lists. Called QGPHY2_PLLVDD in the Design Guide Pg 61. XTAL_AVDD Osc Analog 3.3V from BULK_3V3 Ferrite isolated on Demo Brd. Called XTAL_AVDD in all documents but not in Design Guide Pg 61, 62. References: see pages 61, 62 in the Design Guide, the Demo Board Schematics, and in the Data Sheet pages 136 and the pin lists starting on page 138. This is a total of 11 supply buses to the Switch Chip. As implemented on the Demo Brd, 7 of them have Ferrite power noise filters. There are a number of additional power supply related pins: GPHYS1_RDAC Bias Setting Pin 1.24k Ohm to Gnd Required. GPHYS2_RDAC Bias Setting Pin 1.24k Ohm to Gnd Required. IMP_VOL_REF IMP Interface Voltage Reference Gnd this pin. PLL_AVSS Gnd Tie this pin to Logic Ground. XTAL_AVSS Gnd Tie this pin to Logic Ground. Exposed_Pad Gnd Tie this pin to Logic Ground. As such I do not think that there are any Gnd pins on this part. Its one and only ground connection is through its center Exposed Thermal Pad. The power supply connections to a Switch Chip on the Hub Module are shown in drawing #18. Power-Up Supply Sequencing: --------------------------- The required power supply ramp up sequence is discussed on page 62 and 63 of the Design Guide. They say that failure to properly sequence the supplies will not damage the device but the for "successful" power up the proper supply sequence must be followed and the Reset line used in the proper way. The BCM53128 device requires specific power sequencing between the core and I/O supplies. The device power sequence requires I/O power (3.3V, 2.5V) to come up first, followed by the core power (1.2V). The requirement is that the core power (1.2V) should not be on until the I/O power (3.3V, 2.5V) reaches at least 1.0V. When the core power reaches nominal core voltage (1.2V +-5%), the I/O power should be stable at nominal I/O voltage (3.3V +-5% or 2.5V +-5%). The max ramp-up time for core power 1.2V (from 0V to nominal voltage +-5%) is 2 msec. See page 63. Additionally, for successful power-up, Broadcom recommends that the external hardware reset be active for at least 80 msec after both I/O and core power are stable. See page 63. Note that the required power-up sequence rules out ramping all supplies Volt for Volt because they do not want any power on the core until the I/O is up to at least 1.0V. Thus the Hub Module needs a separate delayed 1.2V Core supply for the Switch Chips. Having a separate 1.2V supply for the Switch Chips was kind of in the cards anyway because clearly it can not be shared with the 1.2V Virtex MGT supply. The power supply current requirements on page 298 of the data sheet appear to be total current from the various voltage buses. I don't know if there are extra start up requirements. 1.2 Volt 1130 mA 2.5 Volt 26 mA (if used) 3.3 Volt 496 mA Hub Module Power Supply Design for Its Switch Chips: ---------------------------------------------------- A separate 12 Amp supply is used for the Switch's 1.2 Volt requirements. This is a GE UDT020A0X1 DCDC Converter and it is enabled after the other DCDC Converters on the Hub Module have all ramped up and specifically after the BULK_3V3 converter has ramped up. The typical draw on this SWCH_1V2 bus for all 3 switch chips is expected to be 3.5 Amps. The typical draw on the BULK_3V3 bus for all 3 switch chips is expected to be 1.5 Amps. The expected heat load for all 3 switch chips is 9.2 Watts. The OVDD power bus to the Switch Chips can be either 2.5 Volt or 3.3 Volt. The OVDD rail powers the GMII - RGMII port I/O pins. On the Hub Module the GMII-RGMII port is not used and for convenience the OVDD rail is powered from the BULK_3V3 supply. The sources of the 11 supply rails to the BCM53128 Switch Chips on the Hub Module are the following: AVDDL 1.2V from SWCH_1V2 DC/DC Converter DVDD 1.2V from SWCH_1V2 DC/DC Converter PLL_AVDD 1.2V from SWCH_1V2 DC/DC Converter GPHY1_PLLDVDD 1.2V from SWCH_1V2 DC/DC Converter GPHY2_PLLDVDD 1.2V from SWCH_1V2 DC/DC Converter AVDDH 3.3V from BULK_3V3 DC/DC Converter OVDD 3.3V from BULK_3V3 DC/DC Converter OVDD2 3.3V from BULK_3V3 DC/DC Converter GPHY1_BAVDD 3.3V from BULK_3V3 DC/DC Converter GPHY2_BAVDD 3.3V from BULK_3V3 DC/DC Converter XTAL_AVDD 3.3V from BULK_3V3 DC/DC Converter Switch Chip ByPass Capacitors and Filter Chokes: ------------------------------------------------ Recall the power supply bypass capacitors recommended by Broadcom. The following is a combination of the BCM53128 Design Guide and the Demo Board Schematic. PLL_AVDD pin (each one) is connected to the Analog 1.2V plane via the ferrite bead. Each pin is bypassed with 22 uFd and 100 nFd capacitors. AVDDL pins are connected to the Analog 1.2V plane and bypassed with a 33 uFd capacitor. Each pin is decoupled with a 1.0 uFd capacitor. DVDD pins are connected to the Digital 1.2V plane and bypassed with a 10 uFd capacitor. Each pin is bypassed with a 100 nFd capacitor. GPHY1_PLLDVDD pins are connected to the Digital 1.2V plane and bypassed with a 10 uFd capacitor. Each pin is bypassed with a 100 nFd capacitor. GPHY2_PLLDVDD pins are connected to the Digital 1.2V plane and bypassed with a 10 uFd capacitor. Each pin is bypassed with a 100 nFd capacitor. XTAL_AVDD pins are connected to the 3.3V plane and bypassed with a 100 nFd capacitor. AVDDH pins are connected to the 3.3V plane and bypassed with a 33 uFd capacitor. Each pin is bypassed with a 1.0 uFd capacitor. OVDD2 pins are connected to the 3.3V plane and bypassed with a 10 uFd capacitor. Each pin is bypassed with a 100 nFd capacitor. OVDD pins are connected to either the 2.5V or 3.3V plane and bypassed with a 10 uFd capacitor. Each pin is bypassed with a 100 nFd capacitor. GPHY1_BAVDD pins are connected to the 3.3V plane and bypassed with a 10 uFd capacitor. Each pin is bypassed with a 100 nFd capacitor. GPHY2_BAVDD pins are connected to the 3.3V plane and bypassed with a 10 uFd capacitor. Each pin is bypassed with a 100 nFd capacitor. On the Hub Module the following 7 supply rails to the switch chips are isolated with a ferrite filter chokes: AVDDL 1.2V PLL_AVDD 1.2V GPHY1_PLLDVDD 1.2V GPHY2_PLLDVDD 1.2V GPHY1_BAVDD 3.3V GPHY2_BAVDD 3.3V XTAL_AVDD 3.3V From the demo board schematics I believe that ferrite chokes with a reactance of 600 Ohms at 100 MHz and a 2 Amp capacity with less than 100 mOhm resistance is adiquate for all 7 of these applications. On the Hub Module the following 4 supply rails to the switch chips are not isolated with a filter choke: DVDD 1.2V AVDDH 3.3V OVDD 3.3V OVDD2 3.3V The BCM53128 Switch Capacitive Coupling: ---------------------------------------- Page 29 of the BCM53128 Design Guide explains that capacitive coupling can be used between pairs of BCM53128 chips for all 3 ethernet speeds. Capacitive coupling works between BMC53128 chips because the BCM53128 uses Voltage Mode Phy technology. Capacitive coupling should also work between the BMC53128 and other parts that use Voltage Mode Phy technology. There are termination schemes that use capacitive coupling between the BCM53128 and parts with older Current Mode Phys parts but this is probably speed dependent and is not needed on the Hub Module. The Micrel KSZ9031 Phys has a voltage mode transmitter connection to the medium so it can probably work with capacitive coupling. The KSZ9031 data sheet talks only about connection with magnetics. Because the KSZ9031 has voltage mode transmitters, the center tap on the chip's side of the transformer should just be tied to ground with a 100 nFd capacitor (i.e. not connection to a power bus). This is just like the transformer connection to the BCM53128 switch chip. The Hub Module design uses capacitive coupling between the 3 BCM53128 Switch chips and between the BCM53128 and the KSZ9031RNX Phys for the Hub's FPGA. Switch Chip Orientation and Port Connections: --------------------------------------------- For all 3 Switch Chips their pin #1 will be in the SE corner. The 1:64 edge is to the East with its Clock, JTAG, 3 LED, and unused IMP connections. The 65:128 edge is to the North. This is Ports 4:7 with Port 4 closest to the Hub's back edge. The 129:192 edge is to the West. This is a bunch of unused IMP stuff and then the bulk of the LED connections. The 193:256 edge is to the South. This is a few LED connections and then Ports 0:3 with Port 3 closest to the back edge of the Hub. Based on the placement of the Switch Chips running B, A, C across the card from front to back and based on the Switch Chip's orientation the following Port assignments are the most straight forward to route. Chip "A" U31 is in the center of the row: Port 0 to J24 Row 8 BI Channel 14 Slot 14 FEX Port 1 to J24 Row 7 BI Channel 13 Slot 13 FEX Port 2 to J24 Row 6 BI Channel 12 Slot 12 FEX Port 3 to J24 Row 5 BI Channel 11 Slot 11 FEX Port 4 to J24 Row 4 BI Channel 10 Slot 10 FEX Port 5 to J24 Row 3 BI Channel 9 Slot 9 FEX Port 6 to Front Panel RJ-45 Connector RJ2 Lower/Left Port 7 to Switch Chip "B" Capacitor Coupled Chip "B" U32 is the "center" Switch Chip: Port 0 not used Port 1 not used Port 2 to Switch Chip "A" Capacitor Coupled Port 3 to Switch Chip "C" Capacitor Coupled Port 4 to This Hub's FPGA Capacitor Coupled Port 5 to J20 Row 3 Update Channel to Other Hub Port 6 to Front Panel RJ-45 Connector RJ3 Lower/Left Port 7 to Front Panel RJ-45 Connector RJ3 Upper/Right Chip "C" U33 is near the cards backplane edge: Port 0 to J24 Row 2 BI Channel 8 Slot 8 FEX Port 1 to J24 Row 1 BI Channel 7 Slot 7 FEX Port 2 to J23 Row 10 BI Channel 6 Slot 6 FEX Port 3 to J23 Row 9 BI Channel 5 Slot 5 FEX Port 4 to J23 Row 8 BI Channel 4 Slot 4 FEX Port 5 to J23 Row 7 BI Channel 3 Slot 3 FEX Port 6 to Front Panel RJ-45 Connector RJ2 Upper/Right Port 7 to Switch Chip "B" Capacitor Coupled Assignment of Ethernet "Magnetics": ----------------------------------- These assignments are given in the file: hub_0_ab_ethernet_line_circuits.txt Assignment of Front Panel RJ-45 Connectors: ------------------------------------------- These assignments are given in the file: hub_0_ab_ethernet_line_circuits.txt Switch Chip Strapping Options: ------------------------------ The BCM53128 switch chip has a large number of options that are controlled at power up by the setup of various strapping pins. Some of these strapping pins are dual purpose, e.g. at power up they enable or disable some option and then once the chip is functioning as a switch the pin surve some other purpose, e.g. an LED pin. There are about 33 of these dual purpose strapping pins: Strapping Pin Function ---------------- ------------------------------------------- CLK_FREQ1/GPIO1 I think that this just sets the Clock Frequency CLK_FREQ0/GPIO0 for the 8051. Take the default 91 MHz. pins 15, 14 CLK_FREQ1 has int PD CLK_FREQ0 has int PU CPU_EEPROM_SEL Pull-up or float, for SPI bus operation pin 18 Pull-down, initial setup from EEPROM We need pull-down to use the EEPROM. HW_FWDG_EN/GPIO7 If this pin is pulled Low at power up then pin 9 Frame Forwarding is Disabled. The default is Low. We want it Hi. Needs a pull-up. HP pulls up. Kit pulls up. LED_MODE(1)/GPIO4 pin 13 Internal pull-down. HP pulls down. Kit allows but does not install PD. LED_MODE(0)/GPIO3 pin 12 Internal pull-up. HP floats. Kit allows but does not install PU. Implies LED_MODE(1:0) = 01 mode value. When LED MODE[1:0] = 01 FE configuration 100M/ACT 10M/ACT DPX/COL PHYLED4 GbE configuration 1G/ACT 10/100M/ACT DPX/COL PHYLED4 IMP_SPD_SEL(1:0) Float and take the default value set pins 50,51 by the internal pull-up-down resistors. IMP_MODE(1)/GPIO6 pin 8 Float and take the default set IMP_MODE(0)/GPIO5 pin 7 by internal pull-up-down resistors. IMP_DUPLEX pin 52 Float and take the default set IMP_LINK pin 54 by internal pull-up-down resistors. IMP_PAUSE_CAP_RX pin 55 Float and take the default set IMP_PAUSE_CAP_TX pin 56 by internal pull-up-down resistors. IMP_VOL_SEL(1:0) IMP Voltage Select Have Internal Pull-Downs pins 48, 49 Must be set Low, Low for 3V3. Both Kit and HP float. ENCLK25_OUT/CLK_25_OUT Float and take the default the default pin 26 Disable of this clock output. Internal PD. ENCLK50_OUT/CLK_50_OUT Float and take the default the default pin 21 Disable of this clock output. Internal PD. EN_EEE Float and take the default to Enable the energy pin 38 efficient etherent features. Has internal PU. Both Kit and HP float. EN_8081_TxRx Float and take the default to Disable the pin 47 8051 from receiving and transmitting packets. Has internal PD. Both Kit and HP float. LED3/EN_GREEN Pull-Up to Enable Green Mode see datasheet pin 172 Pull-Low to Disable Green Mode page 133 Design Guide pg 56 says must pull low EN_Green to use Pseudo-PHY to access internal Sw registers. Note that EN_Green is the same as EN 8051 but see also EN_EEE and EN_8051_TxRx which have or may have to do with 8051 operation. We must Disable the 8051 to use MDC MDIO. Has internal Pull-Down to Disable 8051. Kit Pulls-Up, HP floats. LEDP10/EPROM_TYPE0 pin 181 We need EPROM_TYPE(1:0) = 10 LEDP11/EPROM_TYPE1 pin 182 for the 93C66 type EEPROM so Both have internal Pull-Downs must pull-up LEDP11/EPROM_TYPE1 as per Kit and HP. Loop-Detect This has both a strap pin and an Active pin. LEDP14/LOOP_DETECT_EN pin 186 Low --> Disable Loop Det Hi --> Enable Loop Det Has internal Pull-Down Kit has Pull-Up, HP floats ** ACT_LOOP_DETECT pin 59 Hi to Activate a sweep of Loop Detect. Controlled by a Hub FPGA I/O pin if we ant it ? LEDP15/LOOP_IMP_SEL pin 188 Float and take default Low to Exclude the IMP from Loop Det. Has internal Pull-Down. LEDP18/BC_SUPP_EN pin 191 Low = Default --> Disable rate based broadcast suppression. Hi --> Enable rate-base broadcast suppression See the datasheet page 45. Has int PD. Kit goes Low. HP floats LEDP22/DIS_IMP pin 196 Pull-Up to Disable the IMP Interface. Has internal Pull-Down. We must Disable IMP. Kit Pulls-Up. HP Pulls-Up LEDP23/IMP_DUMB_FWDG_EN pin 197 IMP Port Blocking or Forwarding. Take Default. Has internal Pull-Down. Kit floats, HP floats LEDP26/ENFDXFLOW pin 254 When Hi Enables Full-Duplex Flow Control. See datasheet pate 69. We want the default Hi --> Enable Has internal Pull-Up. Kit floats, HP floats LEDP27/ENHDXFLOW pin 255 When Hi enable Half-Duplex Back Pressure. See datasheet pate 69. We want the default Hi --> Enable Has internal Pull-Up. Kit floats, HO float LEDP30/IMP_TXC_DELAY pin 2 IMP port in RGMII mode Tx & Rx LEDP31/IMP_RXC_DELAY pin 4 Timing Delay. Take Default. We want the default Low Both have internal Pull-Downs. Kit floats, HP floats Resistor Set Pin: ----------------- The RDAC pins, GPHY1_RDAC pin 228 and GPHY2_RDAC pin 96 must be connected to Ground with 1.24k Ohm 1% resistors. Unused Tie Hi, Tie Low, Float Pins: ----------------------------------- There are a number of un-used and no-connect pins on the switch chips. Some of the un-used pins must be tied Hi or Low - others must be left floating. IMP_VOL_REF pin 146 must be tied to Ground. CLK_FREQ0/GPIO0 pin 14 must be left floating CLK_FREQ1/GPIO1 pin 15 must be left floating IMP RGMII Inputs: RXCLK, RXD(3:0), RXCTL (aka RXDV) should all be Pulled-Down These are pins: 144, 154, 152, 151, 150, 149 LOOP_DETECTED pin 58 warning buzzer if a Loop is Detected - float No Connect Pins from BCM Datasheet: NC 5 this is a second MDIO aka MDIO2 NC 6 this is a second MDC aka MDC2 NC 20 NC 28 NC 37 NC 39 NC 41 NC 42 NC 45 NC 46 NC 69 NC 70 NC 99 NC 225 Other unused pins, e.g. LEDCLK 167 HP has small cap to gnd Kit has 100 pFd to gnd LEDDATA 166 HP has small cap to gnd Kit floats TRST pin 36 has an internal Pull-Up but Must be pulled Low for normal operation. See datasheet page 129. Kit has a DNI pull-down. HP has a pull-down. EEPROM Connection to the Switch Chip: ------------------------------------- The BCM53128 contains about 200 internal registers. The initial running state of many of these registers may be set at power up by the contents of an attached EEPROM. Drawing #17 shows the connection of this EEPROM to the BCM53128 switch. This drawing also shows the Clock, Reset, and management connections to the Swtich Chip. The EEPROM is an AT93C66B device with 8 pins. Note that an earlier Atmel device (AT93C66A-2.7) had originally been specified for use with the BCM53128. This EEPROM is powered from the 3V3 bus and 4 of its pins connect to the switch chip: SS slave select pin 160 Low active to the EEPROM SCK SPI Clock pin 163 to the EEPROM MOSI Master Out Slave In pin 164 input to the EEPROM MISO Master In Slave Out pin 161 output from EEPROM All 4 of these pins use series damping resistors, e.g. 22 Ohm. In addition to these 4 signal pins this EEPROM has one power pin and one ground pin and one not connected pin. The ORG pin on the EEPROM is pulled up to set it for 16 bit words. The AT93C66B stores 256 16 bit words. Note that this: SS, SCK, MOSI, MISO interface to the switch chip can be used for either: EEPROM or SPI bus - but not for both. It's use is set for ever more at power up by the CPU_EEPROM_SEL strapping pin, pin 18. Pull it down to enable use with the EEPROM. At power up I think it's basically an issue of setting up the switch's SPI port as a SPI master or a slave. We still have the questions of what content to put into this EEPROM for the Hub application, can all 3 EEPROMs contain exactly the same data, how to program them, and how to change their data if there is a problem in the future. Because of these uncertanties I want to put the EEPROMs on the back side of the Hub PCB and keep them open and easy to change. To facilitate this I will use an SOIC-8 part. This should be part number: AT93C66B-SSHM -B or -T. Both are good DK numbers. This is a normal SOIC with a 3.90 mm wide body, i.e. feet about 6.00 mm wide. Atmel says that its maximum height is 1.75 mm so it fits within the Hub's 2.00 mm side 2 comp height rule. SPI Bus Connection: ------------------- The BCM53128 switch chip includes a serial SPI bus connection. This SPI serial bus allows control and monitoring of the chip through its various registers. The problem is that the SPI serial bus and the EEPROM share the same pins on the device. At power up, a strapping option sets these pins to operate as either an SPI bus interface or as an EEPROM interface. Once this option is set it can not be changed. In the Hub Module we need to use these pins for EEPROM setup of the switch chips. For subsequent control and monitoring of the switch chips from the Hub's FPGA we need to use the MDC MDIO management interface to the device. MDC MDIO Connection to the Hub's Virtex FPGA: ---------------------------------------------- The Hub uses the Switch Chip's MDC MDIO port in slave mode so that an external entity, i.e. the Hub's FPGA, can access registers within the Switch. See the Datasheet starting at page 112 and on page 128. See the Design Guide starting on page 56. MDIO pin 61 Data I/O pin "In slave mode, it is used by an external entity to read/write to the switch registers via the Pseudo-PHY." MDC pin 62 Clock pin "In slave mode (the clock) is sourced by an external entity." Switch Chip Reset: ------------------ The BCM53128 Swtich Chip needs a Reset signal that remains asserted until 80 msec after all of its power supply rails are fully ramped up. We may also want to be able to reset one or all of the Switch Chips from a protected register in the Hub Module's Virtex FPGA. The minimum pulse width of the Reset signal is 1 msec. One must wait 50 msec after the end of the Reset signal before trying to talk to the switch chip via its SPI bus. The Reset signal is pin 17. The Reset signal is Low active. Clock to the Switch Chip: ------------------------- The Hub Module uses an external 25 MHz clock source to provide the clock to all 3 of its Switch Chips. This external clock enters the chip on its XTAL1 pin 34 and the XTAL0 pin 33 is left floating. The clock source for the Switch Chips must be: 25 MHz +- 50 ppm, 50% duty cycle, 1 to 4 nsec edge speed, 3V3 CMOS level, 5 ppm per year, 10 ppm over temperature, less then 100 psec jitter very low jitter in the 5 kHz to 1 MHz band. The Hub Module supplies this clock from a Connor Winfield CWX813-025.0M oscillator with a 6x fanout and series terminator resistors. Switch Chip Functions That Are Not Used on the Hub: --------------------------------------------------- One must understand whether or not any pull-up / pull-downs are needed to not use these functions so that they just sit quietly as good citizens. - Flash Memory, pins: FSO 64, FCSB 65, FSCLK 66, FSI 67 Note that the Flash Memory is for 8051 code only. - JTAG, pins: TDO 22, TDI 23, TCK 24, TMS 25, TRST 36 - IMP Interface - this is the Inband-Management-Port, i.e. a PhysLess port that can operate as: GMII, RGMII, MII, TMII, RxMII or whatever to manage the switch chip. We do not want it at all - thus we must tie DIS_IMP strap pin Hi. The IMP pins - note that the Design Guide says to tie Low unused IMP pins: RXCLK, RXD(3:0), RXCTL but that neither the Kit or HP does this. - List of IMP pins: IMP_VOL_REF 146 must be tied Low IMP_VOL_SEL1 48, IMP_VOL_SEL0 49 IMP_COL 159, IMP_CRS 143 IMP_DUPLEX 52, IMP_LINK 54 IMP_MODE0/GPIO5 7, IMP_MODE1/GPIO6 8 IMP_PAUSECAP_RX 55, IMP_PAUSECAP_TX 56 IMP_SPEED1 50, IMP_SPEED0 51 IMP_GTXCLK 132, IMP_RXCLK 144 IMP_RXD0 150, IMP_RXD1 151 IMP_RXD2 152, IMP_RXD3 154 IMP_RXD4 155, IMP_RXD5 156 IMP_RXD6 157, IMP_RXD7 158 IMP_RXDV 149, IMP_RXER 147 IMP_TXCLK 141, IMP_TXD0 137 IMP_TXD1 136, IMP_TXD2 134 IMP_TXD3 131, IMP_TXD4 130 IMP_TXD5 128, IMP_TXD6 127 IMP_TXD7 126, IMP_TXEN 139 IMP_TXER 140 18-March-2016 Switch Chip Bypass Capacitor and Filter Update: ----------------------------------------------- During the tentative final placement of the 100 or so RCL comps for each Switch chips the following changes were made: PLL_AVDD C01 C02 L01 no-change GPHY1_PLLDVDD C03 C04 L02 no-change GPHY2_PLLDVDD C05 C06 L03 no-change GPHY1_BAVDD C07 C08 L04 no-change GPHY2_BAVDD C09 C10 L05 no-change XTAL_AVDD C11 C12 L06 no-change DVDD SWCH_1V2 C36,C37 C38:C47 no-change AVDDL 1V2 C13,C14 C16:C30 L07 remove from set: C15 C33,C34,C35 AVDDH \ C48, C49, C50 OVDD | BULK_3V3 C51:C62 OVDD2 / C78:C85 remove from set: C67, C68, C76, C77 C63, C64, C65, C66 C69:C75 C86:C90 The components that have been removed from the relatively possitioned component set are now in their own comps file for manual positioning. Other Changes in Switch Comps: R2x19 the clock series terminator has been moved out of the Switch Chip Comps files and into the 25 MHz Enet Clock Generator comps file. C2x91 for the Switch EEPROM has been moved to the top side of the card.