A few minutes of tracking down missing dependencies and 900ish seconds of pegged CPU later and I have a kicad9 install and the project is loaded.

Let's see what's next.

I'm just gonna jump right in without much of an intro, you can read the blog and look at the files in the repo for a summary. The tl;dr is that this is a pre-existing FPGA design with an XCKU5P that has some SI issues and has a planned respin.

First observation: There's a general lack of dedicated return current paths on all of the high speed signals.

Seen here on the SYZYGY connector, almost all of the diff pairs transition to other layers in close proximity to the connector, but there's no return path vias.

The P0.D4 vias for example are going from L1 (ref L2) to L6 (ref L7), but are about 7.7 mm away from the closest ground via that connects the L2 and L7 vias.

I could simulate this, but I can tell just by looking at it that it's not good.

While we're looking at this connector, the connector pads are also 280 μm wide while the differential pair tracks are 150 μm. So we're going to have a significant low-Z bump on the connector.

Possible fixes are

1) Make the pads smaller (will cause solderability issues)

2) Move the reference plane further away and enlarge the trace so the pad is matched (will require a new stackup and also significantly reduce density of routing)

3) Add a ground plane cutout under the connector to reduce coupling to the reference plane and bring the impedance back up to target. This is normally the best choice.

The top layer has ground fill on it, which is something I discourage for a bunch of reasons.

First, pragmatically, it leads to complex shapes that don't mesh well in field solvers and make simulation slower.

Second, you get lots of undesired coupling to top layer signals that will throw off impedance matching. P0.TX2 for example is a de facto grounded coplanar waveguide, while P0.REFCLK is a microstrip for most of its length.

You need to either explicitly design your routing as CPWG (ensuring the top layer return path is continuous, unbroken, and well stitched to the inner layer) or pull the top ground back far enough it won't significantly couple - at which point you're often better off just removing it outright.

Third, there's a lot of isolated copper features that will resonate unless well stitched to inner layers (often not possible in the available space, or will hurt routing density if possible).

For example, the "finger" between P0.D4- and P0.TX3+ is going to strongly couple to both signals, but has no connections elsewhere except to the plane at the east side.

The uncoupled length is about 5.7 mm. Guesstimating an 0.7 velocity factor, I'd expect it to behave as a quarter-wave stub for frequencies with a free space wavelength of (5.7 mm * 4) * 0.7 or 15.96 mm. So we'd expect to see a resonance at 18.78 GHz causing a significant dip in insertion loss.

We'd also probably see a half-wave resonance at half that frequency (9.39 GHz).

And finally, because of how chopped up it gets by signal routes and component pads, top layer ground just doesn't act like a very good return path because it's so discontinuous. There's really not much use for it in most designs.

The DDR is even worse from a return current perspective. Most of the ground pins on the SODIMM are connected to the top layer ground fill only, and the nearest ground vias are at the far north edge of the connector.

The nearest ground via to the B1.D1 layer transition (from L1 to L6) for example is over 8mm away.

On the plus side, the connector pad size and the low-impedance DDR single-ended tracks are well aligned, so there shouldn't be a significant impedance mismatch at the SODIMM itself.

Moving on to the PCIe, we see more of the same issues: giant 0603 coupling caps (0402 or even 0201 would be better from a parasitics perspective) with no ground cutouts for impedance matching, plus L1-L8 vias with no return path provided.

This is particularly bad because there's no stitching vias or ground vias in general near the PCIe fingers.

So the nearest return path for say the TX4_C pair going between the L2 and L7 ground planes is probably the via coming off FPGA ball E1 a whopping 17 mm away.

This is a pretty consistent pattern across the whole board, I'm not going to call out every instance of it (QSFPs etc).

In short, there's a general lack of attention paid to return currents every time a SERDES or DDR signal changes layers, and lack of ground plane voiding on BGA and coupling capacitor pads causing low-Z mismatches.

Ideally, every time a high speed signal changes layers, there should be a return current via - or two for a diff pair - in close proximity. If you want to go all the way, you can do full EM solver simulation of the transition including the coupling between via barrels/pads/planes and vias to optimize the exact clearance from via to plane, spacing from via to via, etc. But just having a dedicated return path is way better than not having one.

See example from one of my boards (not the one being reviewed)

Some more good examples of best practices in this space. This is 5 Gbps QSGMII on a 6-layer board but the same concepts apply to higher layer counts.

In this particular example some of the layer transitions are right under one side of the capacitor using filled via-in-pad, but the same key concepts apply:

1) Any time you have to put a large component pad, in this case an 0402, in an impedance matched signal, add a ground plane cutout to bring the impedance back up to target

2) Provide stitching vias between the two reference planes any time a signal changes between layers referenced to different planes.