Tests from RFC1918 addresses on mlab01.lhr09

[mattmathis]

See the saved query https://console.cloud.google.com/bigquery?sq=581276032543:3e63327785174d50b207ae16597e3b6b

Circumstantial evidence suggests two causes:

1. OAM from cloud avoids a double NAT by routing directly between the cloud VMs
2. Organic tests from within GEN might do the same.

This created a problem for use, because in both cases the client annotations are wrong. Case 1) is discarded normally, but case 2) potentially interferes with using virtual MLab to diagnose GEN and GCloud networking problems.

Case 1 can be partially resolved in two ways:

• 1a Add client annotations to all OAM (generally a good idea)
• 1b Capture the RFC 1918 addresses for all OAM, in addition to the public addresses.
  We need to think more about case 2.

[stephen-soltesz]

Given this is likely related to intra-cloud routing, I'm transferring this issue to k8s-support repo.

[stephen-soltesz]

FYI: @nkinkade

[nkinkade]

@mattmathis: The saved query linked to in the original comment would not work for me without removing node from the field list in the UNION queries. I removed that field from both statements and it ran, but didn't reveal anything interesting (to me).... just rows with machine types, test counts, protocols, etc. Could you provide me with some sample rows showing RFC1918 addresses?

The OAM ndt5 end-to-end testing uses ndt5-client-go. From what I can see, that client does not currently allow sending any additional client metadata to the server.

If what your are seeing is really some sort of internal Google networking magic, it's not immediately clear how we get around that and still run the end-to-end tests from GCP. We could possible run them from some external provider like Linode.

[mattmathis]

I have updated the query to reflect changes in the underlying views. Still investigating.

[nkinkade]

@mattmathis: My understanding is that this might not be an issue after all. Is this correct? If this might have been a false alarm, would you please close this issue? We can always reopen it later.