Everything Instant On

It seems the shared services feature has been forced on in the new portal as of Monday and I need to disable it. Documentation for this procedure is no longer valid: Shared Services.

The situation I'm running into is that I've got 3 VLANs that are bridged together into a single network/subnet. (This is done to permit firewalling between segments of the same subnet.) All 4 APs are now spamming each individual VLAN with MDNS queries twice a second. Under ideal circumstances, that's 8 MDNS queries a second. With 3 VLANs bridged, that's 24 queries a second. With 15 Homekit-capable cameras, that's 360 MDNS answers a second. And they aren't the only thing responding.

WiFi is currently unusable if any of the SSIDs are tied to the bridged VLANs. Configuring dedicated VLANs for WiFi works around the overutilization problem, but breaks MDNS since it isn't replicated between subnets.

From the perspective of the bridge, you can see the AP MACs constantly moving between the VLANs:

I ran into another issue with shared services today...

I initially noticed a spike in CPU on several hosts last night:

I captured packets for just 47 seconds and saw 46,330 requests from a single AP:

I rebooted the AP and the problem went away. Clearly this feature is broken. I don't need it and just want to disable it. Help!

This issue is currently under investigation.

Thanks to Doug who has provided detailed information and logs.

@Sylvain

One additional angle to add - issue manifests in the same way for a common secure home setup of VLANs and a server / box / machine with an iface per VLAN and running a multicast relay. 

https://github.com/alsmith/multicast-relay - here's an example of a popular one that causes an immediate packet storm and brings down the HW and network.
Wireshark pcap looks the same as above, just add thousands of same relayed packets originating from the relay.  

Problem is aggregated (but - I think - not caused) by APs sending multicast packets to individual VLANs using a source IP that doesn't even belong on that VLAN making the relays unable to connect the packet to an interface - hence relaying to all VLANs, including the originating VLAN. 

I changed the relay code to skip all packets orginating from my mgmt VLAN, but that's not a maintainable solution.

I do strongly suggest adding ability to disable this feature altogether.
Couple of arguments:
  - relays usually relay much more than just mDNS, hence you likely won't replace them for people who use them (which is probably most smart home operators that use VLANs).
  - so if they have them - they don't need the additional feature on the AP that only has a potential to cause issues adding no value.

I can observe the same behaviour on my network. After one of the last updates my whole HomeKit setup went belly up (mDNS disabled in Aruba Portal and enabled mDNS repeater on the OPNSense).

Turning off mDNS repeater and using the broken mDNS-Settings in Aruba Portal makes some devices work but not others. 

Are there any news on a fix?

Aruba turned off the shared services feature entirely for my site. Everything has worked well since, even after restoring my bridged VLAN configuration.

Unclear if they intend to re-add the global control to the new UI for everyone else running into issues. They have to realize that not all networks are built the same and this feature is incompatible with some valid designs. So far it seems to not play well with:

• Other mDNS proxies/relays/repeaters/forwarders
• Bridged VLANs

While instant on clearly isn't intended for the most complex networks, there should be a reasonable effort to support valid designs – especially if that effort is "a control to turn a feature on/off". That being said, I'm responsible for a large Aruba AOS 10 wireless deployment with gateways managed by Central at $DAYJOB and it seems to suffer from the same dumbing down of UIs problem.

------------------------------
Doug Hoffman
------------------------------