Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unifi Block Network Access for clients not working #73071

Closed
tux43 opened this issue Jun 5, 2022 · 76 comments · Fixed by Kane610/aiounifi#140, #73716 or #74013
Closed

Unifi Block Network Access for clients not working #73071

tux43 opened this issue Jun 5, 2022 · 76 comments · Fixed by Kane610/aiounifi#140, #73716 or #74013
Assignees

Comments

@tux43
Copy link

tux43 commented Jun 5, 2022

The problem

I was using Home Assistant to block/unblock an iphone by toggling the switch that comes with the Unifi integration. https://www.home-assistant.io/integrations/unifi#block-network-access-for-clients

I upgraded to Home Assistant 2022.6.2 and UniFi OS 1.12.22 and now although the switch is visible when I switch it off it switches back on again (or doesn't actually change state)

image

What version of Home Assistant Core has the issue?

2022.6.2

What was the last working version of Home Assistant Core?

2022.5.5

What type of installation are you running?

Home Assistant Container

Integration causing the issue

UniFi Network

Link to integration documentation on our website

https://www.home-assistant.io/integrations/unifi#block-network-access-for-clients

Diagnostics information

file:///home/bhales/Downloads/config_entry-unifi-20bd161d1ddb7b258529bb054e9f4aa0.json.txt

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

No response

@probot-home-assistant
Copy link

unifi documentation
unifi source
(message by IssueLinks)

@probot-home-assistant
Copy link

Hey there @Kane610, mind taking a look at this issue as it has been labeled with an integration (unifi) you are listed as a code owner for? Thanks!
(message by CodeOwnersMention)

@paulstride
Copy link

Also experiencing this.

For me, the problem is not only blocking network access for clients, but enabling / disabling DPI groups and controlling POE has also stopped working.

If any of these are toggled in the unifi controller, the switch does seem to be updated in home assistant though.

@tux43
Copy link
Author

tux43 commented Jun 6, 2022

If any of these are toggled in the unifi controller, the switch does seem to be updated in home assistant though.

Yes, I found the same behaviour.

@pstoian
Copy link

pstoian commented Jun 6, 2022

+1.
I am experiencing the same issue.
Since the last HA upgrade, I cannot use the switch button to block/unblock a client in UniFi network. But, I noticed that if I am blocking / unblocking a client directly into the UniFi network application - the Home Assistant is displaying the logbooks correctly and the switch is showing the correct state. So, I guess directly from Home Assistant the switch is not working, for now. Hope the developer will fix this bug, - as I am using this feature on daily basis.
Thank you.

HA UniFi Issue

@copart
Copy link

copart commented Jun 6, 2022

Appears to be a duplicate of #70910

@Kane610
Copy link
Member

Kane610 commented Jun 6, 2022

As you stated that it stopped working after 2022.5 at 2022.6.2. Did you by chance upgrade anything in your unifi system? If not. What happens if you downgrade to previous working ha version

@tux43
Copy link
Author

tux43 commented Jun 6, 2022

I also upgraded to UniFi OS 1.12.22 at the same time.

@zee-shany
Copy link

zee-shany commented Jun 7, 2022

Having same issue the toggle buttons don't have any effect. I also upgraded my udmpro to 1.12.22 today, but i'm still on 2022.5.
So I believe its to do with Unifi OS upgrade to 1.12.22.

@jerobins
Copy link

jerobins commented Jun 9, 2022

Following - my HA/unifi setup for blocking devices was working on 2022.5, then I upgraded to 2022.6 yesterday and it stopped working. Getting 'received 401 Unauthorized' on block/unblock calls. Seems to be authenticating fine to get device list, etc. when reviewing the config.

@morb3at
Copy link

morb3at commented Jun 9, 2022

I have the same issue since the last upgrade. the states of the user if have been blocked from unifi updates, but it's not working from HA.

@foleymic
Copy link

foleymic commented Jun 10, 2022

I'm having the same issue - I'm pretty sure it was due to an UNIFI upgrade, not the HA integration. I say this because I was on vacation last week. Yesterday I tried to block my son's devices and noticed this behavior. HA was still on 2022.0 - I hadn't upgraded it in awhile. My UDM Pro was configured for auto upgrades and I noticed a new version of Unifi OS came out 9 days ago (https://community.ui.com/releases/UniFi-OS-Dream-Machines-1-12-22/851bdc97-fc39-40ef-bd71-786766512c58). Unifi Network version was also not upgraded - not sure which version I was on, but I did upgrade it manually yesterday hoping the issue would be resolved. I also upgraded HA to 2022.6.4, but still no luck. I am going to attempt to downgrade Unifi OS next. I will provide an update soon.

[[Update]]
Unfortunately, I had no luck in downgrading my UDM Pro firmware. Well actually, I was able to downgrade the firmware, but I couldn't get the unifi network to work so I had to upgrade it back.

@paulstride
Copy link

[[Update]] Unfortunately, I had no luck in downgrading my UDM Pro firmware. Well actually, I was able to downgrade the firmware, but I couldn't get the unifi network to work so I had to upgrade it back.

Same, I downgraded to the previous firmware, but couldn't get the unifi web interface to work, so had to upgrade again

@zee-shany
Copy link

Has anyone tried removing the home assistant integration and adding it back?

@morb3at
Copy link

morb3at commented Jun 12, 2022

Yes i removed the integration twice with no luck, but there is one thing has been changed before the name of the entity was switch.name now i saw it switch.name_block

@jyavenard
Copy link
Contributor

Has anyone tried removing the home assistant integration and adding it back?

Yes many times.
i also played with the permissions settings as things have changed recently in Unifi OS (you now need to create a new profile and make the account use that profile) with no luck.

@stonith
Copy link
Contributor

stonith commented Jun 14, 2022

I just upgraded to UDM 1.2.22 on my UDMP and can no longer toggle clients. I've also tried reinstalling the integration without luck.

I tried to enable DEBUG logging but it didn't provide any useful info.

@foleymic
Copy link

FYI, I have confirmed that the REST API to block and unblock a client in UNIFI still works. I tested the below in node red:
image

@johntdyer
Copy link
Contributor

I am getting a 403 back from the UDM

022-06-14 11:17:51 DEBUG (MainThread) [aiounifi.controller] https://192.168.100.1:443/proxy/network/api/s/default/cmd/stamgr
2022-06-14 11:17:51 DEBUG (MainThread) [aiounifi.controller] 403 application/json <ClientResponse(https://192.168.100.1:443/proxy/network/api/s/default/cmd/stamgr) [403 Forbidden]>
<CIMultiDictProxy('Vary': 'Origin', 'X-DNS-Prefetch-Control': 'off', 'X-Frame-Options': 'SAMEORIGIN', 'Strict-Transport-Security': 'max-age=15552000; includeSubDomains', 'X-Download-Options': 'noopen', 'X-Content-Type-Options': 'nosniff', 'X-XSS-Protection': '1; mode=block', 'Accept-Ranges': 'bytes', 'Content-Type': 'application/json; charset=utf-8', 'X-Response-Time': '2ms', 'Content-Length': '32', 'Date': 'Tue, 14 Jun 2022 15:17:51 GMT', 'Connection': 'keep-alive')>

@johntdyer
Copy link
Contributor

I am using an admin and I still always get this 403 error when I try to toggle internet for a device

@stonith
Copy link
Contributor

stonith commented Jun 14, 2022

I see the same 403 forbidden error as well when I toggle a device on/off in HASS. I didn't see it before because aiounifi debug is a fire hose.

configuration.yaml

logger:
  default: info
  logs:
    aiounifi: debug
    homeassistant.components.unifi: debug
    homeassistant.components.device_tracker.unifi: debug
    homeassistant.components.switch.unifi: debug

@foleymic
Copy link

foleymic commented Jun 14, 2022

Thank you @johntdyer and @stonith. I should have noticed you had aiounifi set to debug. I was only setting homeassistant.components.unifi: debug so I didn't see the 403. I am now seeing it as well.

@johntdyer
Copy link
Contributor

So I attached a debugger and got more data from the response , seems the response in the aiounif client when triggering the switch is a b'{"message":"Invalid CSRF Token"}'.... not sure why...

@stonith
Copy link
Contributor

stonith commented Jun 14, 2022

I am not seeing it as well.

You are NOT seeing it or NOW seeing it?

@foleymic
Copy link

I am not seeing it as well.

You are NOT seeing it or NOW seeing it?

Sorry about that. I corrected it. I am NOW seeing it.

@stonith
Copy link
Contributor

stonith commented Jun 15, 2022

I setup mitmproxy to inspect the requests. The POST request to toggle the switch uses the same x-csrf-token and cookie token header as all the successful GET requests home assistant makes for retrieving status. Looking for something to compare to that's still working.

UPDATE: looks like home assistant isn't updated the x-csrf-token from the login request. If I take the x-csrf-token from the login response and replay the POST request with it, the request succeeds. Even after a restart of home assistant it seems to use an old x-csrf-token.

@corecoding
Copy link

I am experiencing the same thing. My HA has a section to control the kids internet. It stopped working, for about two weeks now. I tried removing/adding the integration, as well as resetting password in UniFi, etc.

@stephack
Copy link

Same here. Upgraded to 2022.6.7 and the switches worked. After a few hours I received the 401 errors in my logs.

@markus99
Copy link

Running 2022.6.7 as well and not able to even setup the integration (removed it after I started receiving the 401 errors as well). Running an updated UDMPro.

Logger: aiohttp.server
Source: components/unifi/config_flow.py:112
First occurred: 11:00:37 AM (1 occurrences)
Last logged: 11:00:37 AM

Error handling request
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/aiohttp/web_protocol.py", line 435, in _handle_request
    resp = await request_handler(request)
  File "/usr/local/lib/python3.9/site-packages/aiohttp/web_app.py", line 504, in _handle
    resp = await handler(request)
  File "/usr/local/lib/python3.9/site-packages/aiohttp/web_middlewares.py", line 117, in impl
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/security_filter.py", line 60, in security_filter_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/forwarded.py", line 100, in forwarded_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/request_context.py", line 28, in request_context_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/ban.py", line 79, in ban_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/auth.py", line 220, in auth_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/view.py", line 137, in handle
    result = await result
  File "/usr/src/homeassistant/homeassistant/components/config/config_entries.py", line 205, in post
    return await super().post(request, flow_id)
  File "/usr/src/homeassistant/homeassistant/components/http/data_validator.py", line 62, in wrapper
    result = await method(view, request, *args, **kwargs)
  File "/usr/src/homeassistant/homeassistant/helpers/data_entry_flow.py", line 109, in post
    result = await self._flow_mgr.async_configure(flow_id, data)
  File "/usr/src/homeassistant/homeassistant/data_entry_flow.py", line 260, in async_configure
    result = await self._async_handle_step(
  File "/usr/src/homeassistant/homeassistant/data_entry_flow.py", line 335, in _async_handle_step
    result: FlowResult = await getattr(flow, method)(user_input)
  File "/usr/src/homeassistant/homeassistant/components/unifi/config_flow.py", line 112, in async_step_user
    sites = await controller.sites()
  File "/usr/local/lib/python3.9/site-packages/aiounifi/controller.py", line 145, in sites
    sites = await self.request("get", url=url)
  File "/usr/local/lib/python3.9/site-packages/aiounifi/controller.py", line 277, in request
    return await self._request(method, path, json, url)
  File "/usr/local/lib/python3.9/site-packages/aiounifi/controller.py", line 323, in _request
    raise LoginRequired(f"Call {url} received 401 Unauthorized")
aiounifi.errors.LoginRequired: Call https://192.168.###.1:443/proxy/network/api/self/sites received 401 Unauthorized

This log is from trying to re-setup the Unifi Network integration in HA. Receive this error in the UI while trying to re-setup the integration (after removing, rebooting).

image

@stonith
Copy link
Contributor

stonith commented Jun 24, 2022

I got the same error everyone else got and it's permanent. The aiounifi version changed as well and there were some other changes. My manual change for 2022.6.6 seemed to have worked better. I'll try and debug this weekend.

@Kane610
Copy link
Member

Kane610 commented Jun 24, 2022

Make sure to update to the latest versions of unifi os and the controller

@stonith
Copy link
Contributor

stonith commented Jun 25, 2022

Session expiry never gets retried because can_retry_login is was accidently removed in Kane610/aiounifi#140. I submitted a PR here Kane610/aiounifi#147

Manual workaround:

  • edit /usr/local/lib/python3.9/site-packages/aiounifi/controller.py
  • add to line 131
    async def login(self) -> None:
        """Log in to controller."""
        if self.is_unifi_os:
            url = f"{self.url}/api/auth/login"
        else:
            url = f"{self.url}/api/login"
        auth = {
            "username": self.username,
            "password": self.password,
            "remember": True,
        }

        await self._request("post", url=url, json=auth)
        self.can_retry_login = True # Add this
        if (
            (response := self.last_response) is not None
            and response.status == HTTPStatus.OK

@zee-shany
Copy link

zee-shany commented Jun 25, 2022

I've set this self.can_retry_login = True however i still get same error.

Logger: homeassistant.components.websocket_api.http.connection
Source: components/unifi/switch.py:344
Integration: Home Assistant WebSocket API (documentation, issues)
First occurred: 7:20:05 PM (5 occurrences)
Last logged: 7:38:49 PM


[140430812752432] Call https://192.168.0.1:443/proxy/network/api/s/default/cmd/stamgr received 401 Unauthorized
[140430554828960] Call https://192.168.0.1:443/proxy/network/api/s/default/cmd/stamgr received 401 Unauthorized
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/websocket_api/commands.py", line 193, in handle_call_service
    await hass.services.async_call(
  File "/usr/src/homeassistant/homeassistant/core.py", line 1704, in async_call
    task.result()
  File "/usr/src/homeassistant/homeassistant/core.py", line 1741, in _execute_service
    await cast(Callable[[ServiceCall], Awaitable[None]], handler.job.target)(
  File "/usr/src/homeassistant/homeassistant/helpers/entity_component.py", line 204, in handle_service
    await service.entity_service_call(
  File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 680, in entity_service_call
    future.result()  # pop exception if have
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 964, in async_request_call
    await coro
  File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 717, in _handle_entity_call
    await result
  File "/usr/src/homeassistant/homeassistant/components/unifi/switch.py", line 344, in async_turn_off
    await self.controller.api.clients.block(self.client.mac)
  File "/usr/local/lib/python3.9/site-packages/aiounifi/interfaces/clients.py", line 25, in block
    return await self.controller.request(
  File "/usr/local/lib/python3.9/site-packages/aiounifi/controller.py", line 277, in request
    return await self._request(method, path, json, url)
  File "/usr/local/lib/python3.9/site-packages/aiounifi/controller.py", line 323, in _request
    raise LoginRequired(f"Call {url} received 401 Unauthorized")
aiounifi.errors.LoginRequired: Call https://192.168.0.1:443/proxy/network/api/s/default/cmd/stamgr received 401 Unauthorized



@zee-shany
Copy link

zee-shany commented Jun 25, 2022

I've temporarly put in place this node red reload integration node, if anyone is intrested.

unifi integration id you can get from ~config/config_entries/entry folder

image

@stonith
Copy link
Contributor

stonith commented Jun 25, 2022

I've set this self.can_retry_login = True however i still get same error.

That's weird. Both my HA installs no longer have the error and I can see it retry properly. Are you sure you edited the right self.can_retry_login (there's mutiple)? Also, it may require a restart of HA and not just a reload of the integration.

@jerobins
Copy link

I'm seeing the same behavior as @zee-shany. I only changed the file above (cut and pasted the full path) and restarted HA. The integration will not start, 401 unauthorized. I have not tried removing and re-adding it, was trying to avoid that.

@stonith
Copy link
Contributor

stonith commented Jun 26, 2022

Sorry all, I mistakenly removed the can_retry_login attribute in the original fix which broke retries. The subsequent fix for the default session retry will only retry once since it's not reset like originally, so third time's a charm and putting back the attribute in the login function should fix it properly as per updated #73071 (comment) I haven't tested it over mutiple session expiries yet though.

@stonith
Copy link
Contributor

stonith commented Jun 27, 2022

Haven't had any 401 errors since. Can anyone else confirm it's fixed for them with the latest fix?

@zee-shany
Copy link

i can confirm, I've not had 401 errors too.
Thanks for the fix.

@tux43
Copy link
Author

tux43 commented Jun 27, 2022

I’m still getting them. Can you please confirm the fix on 2022.6.7 to ensure I have made the right modifications.

@stonith
Copy link
Contributor

stonith commented Jun 27, 2022

I’m still getting them. Can you please confirm the fix on 2022.6.7 to ensure I have made the right modifications.

See this comment: #73071 (comment)

@jerobins
Copy link

jerobins commented Jun 27, 2022 via email

@zee-shany
Copy link

My HA is 2022.6.7, so far so good, devices are turning off and on as expected.

@tux43
Copy link
Author

tux43 commented Jun 29, 2022

I’m still getting them. Can you please confirm the fix on 2022.6.7 to ensure I have made the right modifications.

See this comment: #73071 (comment)

I tried this again and second time lucky, it is operational now. I think I had a tab rather than a space on line 131 which was my issue.

@Kane610
Copy link
Member

Kane610 commented Jun 29, 2022

The final fix will be available with the 2022.7 release next week. Beta is coming out later today if you want to try it out

@jamespeek
Copy link

It looks like this fixed the authentication, but it appears to introduce a race condition where toggling multiple Unifi devices to block or unblock at the same time fails.

My scenario is I have a scene with multiple devices set to block (and another where they are unblocked). I've found that if I schedule this to run at a specific time, it runs, but only one of the devices toggles state.

I suspect the problem is that the first device tries to toggle, but the token has expired, so gets a 401. It then re-authenticates, retries and succeeds. But the other devices must not retry and and fail to toggle.

My work around for this for the time being is to trigger a single unifi toggle in the script, wait a few seconds, then trigger the scene change.

To recreate, you'll need a scene with multiple unifi switches set to toggle either on or off. You should find this will work if you toggle manually. To get it to fail you'll need to leave it 12 hours (or whatever the token expiry is) and then toggle the scene. You should see that one of the devices defined toggles and the rest do not.

@zee-shany
Copy link

@jamespeek the issue of toggling multiple unifi devices from block/unblock state has been there since the longest time in my experience, at least since an year back. Many users have reported that but no fix so far and mostly everyone is using the workaround i.e putting a delay of few seconds before automation blocks/unblocks a device.

@jamespeek
Copy link

jamespeek commented Jul 21, 2022

@zee-shany I wasn't seeing this before the api authentication issues associated with this ticket; previously using scenes to unblock/block devices in bulk was working pretty flawlessly for me. I guess the other solution would be to have a scheduled task that periodically pings something on the unifi api side, to keep the access token alive (or refresh it out of band, if need be).

@jyavenard
Copy link
Contributor

Same. Having to toggle on off repeatedly a few times with 5-6s pause in between has been there a long time.
It's the same with unifi network app (both phone and Web)

@jerobins
Copy link

My experience was the same as @jamespeek. I had a group of devices and was turning on/off [block/unblock] the whole group without incident until the 2022.6 update.

@zee-shany
Copy link

Thought I'd check here, my kids devices (iphones) are not getting unblock in the morning (blocks at 10:00pm and set to unblock at 5:30am through automation with delays of 5s). I've tried removing unifi integration and adding back. The toggle switch for their device works fine if the block is for short duration for e.g 5-10 mins.
Anyone facing similar issue?
Thanks

@jerobins
Copy link

jerobins commented Aug 9, 2022 via email

@zee-shany
Copy link

Sounds like WiFi privacy is still turned on for the devices. Have to disable private addressing for your home network on each iOS device.

On Tue, Aug 9, 2022, 2:31 AM zee-shany @.> wrote: Thought I'd check here, my kids devices (iphones) are not getting unblock in the morning (blocks at 10:00pm and set to unblock at 5:30am through automation with delays of 5s). I've tried removing unifi integration and adding back. The toggle switch for their device works fine if the block is for short duration for e.g 5-10 mins. Anyone facing similar issue? Thanks — Reply to this email directly, view it on GitHub <#73071 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABXHH7YFPFNKHLSTR4IMFDVYH3LZANCNFSM5X46QSYA . You are receiving this because you are subscribed to this thread.Message ID: @.>

Hmm, I can confirm wifi privacy is turned off, their phone can't connect to network with a random mac address.

@jamespeek
Copy link

@zee-shany

Thought I'd check here, my kids devices (iphones) are not getting unblock in the morning (blocks at 10:00pm and set to unblock at 5:30am through automation with delays of 5s). I've tried removing unifi integration and adding back. The toggle switch for their device works fine if the block is for short duration for e.g 5-10 mins. Anyone facing similar issue? Thanks

This sounds like the same issue I had; my solution was to make my unblock script trigger an unblock event for one device first, wait 10 seconds, then trigger the unblock for all devices.

Not directly related to home assistant, but with your kid's iPhones; don't you find that blocking wifi just means the devices switch to cellular data instead or are they purely wifi? I couldn't find anyway to schedule cell access via the parental controls other than just blocking access apps at certain times of the day.

@zee-shany
Copy link

@jamespeek Thanks for your response.
I already have node red automation to block/unblock after 5seconds for each device. the issue i have is devices don't get unblock if the block period is for long hours. I'm thinking i'll try HA automation instead of node-red and see if i can isolate the issue.

well, luckily for me, my kids aren't given data pakage as yet. so thats not the worry at the moment for me. however I've read about Circle having option to block access over cellular by installing a VPN profile on iphones. but that means you'll have to add another layer of control as compared to just playing with Unifi and HA.

@github-actions github-actions bot locked and limited conversation to collaborators Sep 9, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.