Instant On - Wireless

Expand all | Collapse all

Problem AP22 and 2.3.1

  • 1.  Problem AP22 and 2.3.1

    Posted 07-12-2021 04:41 PM
    Having major problems with 2.3.1.  Macbooks seem to have an issue where they are connected just fine, but can't contact their gateway on occasion.  If I get 2 pings going, one to a local server, and one out to the internet, the one to the local server doesn't drop. However the one out to the internet will drop for maybe 20-30 times in a row and the magically come back. 

    These are 3 AP22s connects to a 1930 switch, with a SonicWall as a gateway. I have confirmed the SonicWall isn't blocking anything.  The same Macbooks I can connect to the ISP provided wifi with 0 issues. So it's nothing like that. 

    I don't know, these AP22s have been solid, and now they are just flaking out everywhere even though they are up to date and I've rebooted them.  I'm always at max signal on the macbooks. Real worlds transfer speeds are above 300mbps. They just suddenly can't communicate with the gateway, yet an iphone right next to it, connected to the same SSID, is sitting there running speed tests without any drops, all while the macbook can't get to the gateway.

    I've turned off Wifi6.  Not sure what else to check.


  • 2.  RE: Problem AP22 and 2.3.1

    Posted 07-13-2021 10:24 AM
    Hi,

    can you confirm that turning off WiFi6 knob helped resolving the issue?
    Also, if you can confirm your MacOS version and Macbook type that'll be appreciated.

    Thanks,

    ------------------------------
    Jul.
    ------------------------------



  • 3.  RE: Problem AP22 and 2.3.1

    Posted 07-13-2021 10:36 AM
    2018 Macbook Pro.  Fully up to date Big Sur.  Turning off WiFi6 had no effect.

    I'm still testing, but so far it seems to only happen on an SSID that has both 2.4 and 5Ghz enabled. Last night I tested on an SSID with only 5Ghz, with WiFi6 ON and optimize for video streaming ON and it didn't drop at all. I'm back on a combined SSID now to verify issue returns.  If it does, next I will try 2.4Ghz only to see how that is.

    Thanks,

    Rob


  • 4.  RE: Problem AP22 and 2.3.1

    Posted 07-13-2021 01:10 PM
    I can now confirm that the problem exists on SSIDs with both 2.4Ghz and 5Ghz enabled. Leaving all other settings the same, simply removing 2.4Ghz from the SSID resolves the issue.  Soon I will test with 2.4Ghz only to see if that has the same issue as the combined.


  • 5.  RE: Problem AP22 and 2.3.1

    Posted 08-09-2021 09:35 AM
    Just adding this is still going on. I even added a 4th AP22 to rule out a hardware problem and while connected to it the problem happens. The problem doesn't happen while connected to AT&Ts provided Harris CPE. 

    It also happens on all bands, it's not isolated to 2.4Ghz. Even on a 5Ghz only SSID, with Wifi6 and media optimizations disabled, it still happens, just not as frequently.  It's obvious streaming something like HBOMax which tends not to buffer as much. You'll be watching and then it suddenly stops and says can't connect. I look at my pings and sure enough, pings to a local server are fine, but pings out the gateway drop. I can't even ping the local gateway (SonicWall) itself during this so it's not an internet problem. This persists for perhaps 15 seconds then resumes and is fine again for a couple hours usually before it happens again. idk, something is definitely wrong. It doesn't happen on wired connections. It doesn't happen connecting with WiFi to the AT&T CPE. It only happens while connected to AP22s.


  • 6.  RE: Problem AP22 and 2.3.1

    Posted 08-12-2021 11:39 AM

    Interesting (and frustrating) problem! 

    You initially said this was only affecting MacBooks, while iPhones were okay. Does that still hold true, or have you discovered otherwise? 



    ------------------------------
    Ethan
    ------------------------------



  • 7.  RE: Problem AP22 and 2.3.1

    Posted 08-12-2021 04:03 PM
    The problem is that it's hard to test. It can go several hours with no issue then it hits. The MacBook is used heavily for streaming video so it's obvious when it happens. None of the other devices are used in such a way where they could easily detect a drop. When the drop happens, it's usually for about 45 seconds. Enough to stop streams.  I haven't noticed it on other devices, but I can't rule it out either.


  • 8.  RE: Problem AP22 and 2.3.1

    Posted 08-12-2021 04:11 PM

    Ugh. These types of issues are tough to diagnose if you can't be onsite and don't have remote monitoring. 

    We've been seeing similar issues with our provider here in Western Canada, who uses the same equipment as Comcast. It's their modem/routers (XB6&7) that are having the issues. 

    Have you double checked to be sure there's no accidental QOS / Application blocking or similar settings in the router and APs that might be choking the video feeds? Don’t forget that with an errant click you can neuter application & traffic types.

    Again - is it only Video streams that are affected? 

    Are you familiar with Wireshark? It would be interesting to fire it up when there’s an issues to see what the frames and packets are doing. 

    ------------------------------
    Ethan
    ------------------------------



  • 9.  RE: Problem AP22 and 2.3.1

    Posted 08-12-2021 04:22 PM
    Yes I'm familiar with WireShark, it's hard to get the timing though. I would have to sit and sit waiting.  There's no QoS or bandwidth mgmt in use. It's a synchronous 1Gbps circuit. I leave 3 pings running on the MacBook.

    1 to local gateway (sonicwall)
    1 to 8.8.8.8
    1 to a local server

    During the event. The last one, to a local server in the same subnet, that never drops. It's always fine. During the event, all pings to the local gateway, and then obviously out to the internet from there, fail. Even with QoS or BWM, it still wouldn't drop all pings for 45 seconds to the local gateway.


  • 10.  RE: Problem AP22 and 2.3.1

    Posted 08-12-2021 04:29 PM

    All three pings are being transmitted over wifi?
    If so, I don’t think you have a wifi issue, you have a routing issue. Do you have another router you can drop in place of the sonic wall?

    ------------------------------
    Ethan
    ------------------------------



  • 11.  RE: Problem AP22 and 2.3.1

    Posted 08-12-2021 06:39 PM
    Yes all 3 pings are being transmitted over wifi. The macbook is not wired up, it's only wifi. However, I disagree with the routing problem assessment. The SonicWall I had was a TZ400 and I had the problem. I recently replaced it with a newer TZ470 and still have the problem. There are no other routers.  All 4 APs connect to an Aruba 1930 switch which uplinks to the SonicWall. All other wired devices on the same 1930 switch have no issues, ever. It's a flat network, no subnets or VLANs. It's not the Macbook itself, as the problem never, not even once happens, while I'm connected to directly to the ISP CPEs wifi. It only happens while connected to these Aruba AP22s. It also never happened before. One of the recent updates is when it triggered.


  • 12.  RE: Problem AP22 and 2.3.1

    Posted 08-13-2021 02:17 AM

    You have me worried now. I'm going to pull an AP22 off the shelf and start testing. Better that we discover an issue in the lab instead of a client's in-production network. 

    If I can replicate the issue I'll report back. 

    Just thinking: The ping to your local server is over layer 2. It never drops, right? It's only the layer 3 pings & traffic that are dying. Doesn't that squarely point back to the router? WiFi only exists in layers 1&2. If the wifi was the problem, why aren’t your pings to the local server also dying? They’re literally running over the same layer 2 network.

    We know traffic is flowing most of the time - so what could be intermittently disrupting / corrupting traffic? 



    ------------------------------
    Ethan
    ------------------------------



  • 13.  RE: Problem AP22 and 2.3.1

    Posted 08-13-2021 08:00 AM
    The gateway itself is also responding at layer 2. Pings to the gateway fail. Now the ping out to 8.8.8.8 would be operating at layer 3. 

    Client IP:  192.168.1.50
    Gateway: 192.168.1.254
    Local server: 192.168.1.10

    Pings from that client, to either the gateway or local server operate at layer 2 only.  There is no routing in this scenario, it's all layer 2.  The only time layer 3 routing is involved is when I'm pinging 8.8.8.8.

    So if a ping fails to 192.168.1.254, and not to 192.168.1.10, that has nothing to do with routing, and everything to do with the APs.  That's why I disagreed with the routing issue assessment.

    I'll add that by this point, I'm using a new SonicWall and a new AP22 in the location. So neither of those are the culprit in terms of possible faulty hardware. The only other item that hasn't been replaced is the 1930 switch, but wired clients on it never have an issue, so I don't believe it is the culprit.

    What's interrupting traffic is a bug with the APs.


  • 14.  RE: Problem AP22 and 2.3.1

    Posted 08-22-2021 05:51 AM
    If ping to local server 192.168.1.10, then I think it is not wifi issue. If you have same result when ping to gateway and local server, then it will be wifi issue

    ------------------------------
    DENNY .
    ------------------------------



  • 15.  RE: Problem AP22 and 2.3.1

    Posted 08-27-2021 04:00 PM
    I am sorry but that's just not true. I have witnessed similar bugs on other deployments with different gear. The local gateway is the same thing as a local server. So I completely disgree with that assessment.

    I also have more notes now.

    1. The issue affects all devices. Not just Macbooks. It I have verified that Apple TVs, Macbook Pros, Macbook Airs, and a Windows Laptop all have the issue.
    2. It's not DHCP Renewal related. The issues happens even when statically assigned.
    3. I've maually channelized the network, dropping 40Mhz wide channels instead of 80, and avoiding all DFS channels. The issue still happens.

    So I'm at a loss now. This wasn't there with previous firmware.  

    Pings are consistent as well.  After running overnight:

    Pings to local server 0.5% loss
    Pings to Gateway 1% loss
    Pings to Internet 1.5% loss.

    So the loss is only 1% to the gateway but what sucks is almost all of that loss is in a chunk of 60 dropped packets in a row which breaks connections, streams, VPNs.  This happens I would say on average about once per hour to all clients connected to these 4 AP22s.  None of this is witnessed when just connecting to the ISPs Harris device.

    New firewall, new APs, none of that solved it.  All other wired machines on the same 1930 switch can ping the gateway all night long without a single dropped packet. So it isn't the wired network either. All 4 AP22s are home run to the 1930. No mesh is in use. One of these recent firmware updates triggered this.


  • 16.  RE: Problem AP22 and 2.3.1

    Posted 08-27-2021 04:08 PM
    Oh and when it happens, it happens simultaneously to all clients. At least all clients on the same AP.  I need to do some more testing simultaneously from multiple APs to see if it syncs up with all APs, or if it happens individually per AP, affecting only those clients on that AP.  I know for fact that at a minimum, when it happens to 1 AP, it affects all clients at the same time, in the same way, who are associated with that AP.


  • 17.  RE: Problem AP22 and 2.3.1

    Posted 08-29-2021 04:27 PM
    What SonicWALL firmware are you using?

    ------------------------------
    Stephan Gladiadis
    ------------------------------



  • 18.  RE: Problem AP22 and 2.3.1

    Posted 08-30-2021 03:26 PM
    SonicOS 7.0.0-R906 on a TZ470.  But the problem started before I swapped out the older TZ400 that had different firmware. So I don't think it's SonicWall related at all for several reasons.

    1. The SonicWall was upgraded with a new device and new firmware, didn't fix this issue.
    2. There is never an issue on the wired network. You can ping the SonicWall at 0% loss all day from any wired endpoint including endpoints on the same 1930 switch as the AP22s.
    3. It's a flat network with a single subnet and no VLANs.
    4. There are never any issues listed in the SonicWall logs that could even remotely be related to this.