user_ldap acceptance tests failing #37428

phil-davis · 2020-05-21T11:52:08Z

e.g. https://drone.owncloud.com/owncloud/user_ldap/2523/48/11

--- Failed scenarios:

    /var/www/owncloud/testrunner/apps/user_ldap/tests/acceptance/features/cliProvisioning/userSync.feature:15
    /var/www/owncloud/testrunner/apps/user_ldap/tests/acceptance/features/cliProvisioning/userSync.feature:25
    /var/www/owncloud/testrunner/apps/user_ldap/tests/acceptance/features/cliProvisioning/userSync.feature:61
    /var/www/owncloud/testrunner/apps/user_ldap/tests/acceptance/features/cliProvisioning/userSync.feature:74
    /var/www/owncloud/testrunner/apps/user_ldap/tests/acceptance/features/cliProvisioning/userSync.feature:87

e.g.

  Scenario: admin deletes ldap users and syncs only one of them       # /var/www/owncloud/testrunner/apps/user_ldap/tests/acceptance/features/cliProvisioning/userSync.feature:15
    When the administrator deletes user "user0" using the occ command # OccUsersGroupsContext::theAdministratorDeletesUserUsingTheOccCommand()
    And the administrator deletes user "user1" using the occ command  # OccUsersGroupsContext::theAdministratorDeletesUserUsingTheOccCommand()
    Then user "user0" should not exist                                # FeatureContext::userShouldNotExist()
    And user "user1" should not exist                                 # FeatureContext::userShouldNotExist()
    When LDAP user "user0" is resynced                                # UserLdapGeneralContext::ldapUserIsSynced()
    Then user "user0" should exist                                    # FeatureContext::userShouldExist()
      User 'user0' should exist but does not exist
      Failed asserting that false is true.
    And user "user1" should not exist                                 # FeatureContext::userShouldNotExist()

~~I think that some code changes related to the way the acceptance tests keep track of createdUsers are still not working nicely in the LDAP user case.~~

Needs investigation and fixing!

These fails started happening 3 days ago - after core PR #37398 was merged.

The text was updated successfully, but these errors were encountered:

phil-davis · 2020-05-22T05:59:11Z

Single user sync seems to not be finding the specified user.
I added some extra code in owncloud/user_ldap#557 to see what the user:sync command outputs:

https://drone.owncloud.com/owncloud/user_ldap/2525/48/11

  Background:                                                                               # /var/www/owncloud/testrunner/apps/user_ldap/tests/acceptance/features/cliProvisioning/userSync.feature:8
    Given these users have been created with default attributes and without skeleton files: # FeatureContext::theseUsersHaveBeenCreatedWithDefaultAttributesAndWithoutSkeletonFiles()
      | username |
      | user0    |
      | user1    |

  @skipOnOcV10.3
  Scenario: admin deletes ldap users and syncs only one of them       # /var/www/owncloud/testrunner/apps/user_ldap/tests/acceptance/features/cliProvisioning/userSync.feature:15
    When the administrator deletes user "user0" using the occ command # OccUsersGroupsContext::theAdministratorDeletesUserUsingTheOccCommand()
    And the administrator deletes user "user1" using the occ command  # OccUsersGroupsContext::theAdministratorDeletesUserUsingTheOccCommand()
    Then user "user0" should not exist                                # FeatureContext::userShouldNotExist()
    And user "user1" should not exist                                 # FeatureContext::userShouldNotExist()
    When LDAP user "user0" is resynced                                # UserLdapGeneralContext::ldapUserIsSynced()
      │ ldapUserIsSynced success. stdErr output was: 
      │ ldapUserIsSynced success. stdOut output was: Searching for user0 ...
      │ Exact match for user user0 not found in the backend.
      │ Deleting accounts:
      │ user0, ,  (no longer exists in the backend)
      │ 
      │ 
      │ 
    Then user "user0" should exist                                    # FeatureContext::userShouldExist()
      User 'user0' should exist but does not exist
      Failed asserting that false is true.
    And user "user1" should not exist                                 # FeatureContext::userShouldNotExist()

@mrow4a since PR #37398 was merged to core master, user:sync tests in user_ldap are failing - single user sync is not finding a single user any more, e.g. user0

The tests pass when run against core "latest" which is the core 10.4.1 release tarball, bbut fail when run against the daily-master-qa tarball, which is built each day from the core git master branch.

mrow4a · 2020-05-22T06:29:32Z

It happens with master but not with 10.4.X? Ok maybe I missed something as I never test apps against unstable core (never master, only on tags)

phil-davis · 2020-05-22T06:33:23Z

It happens with master but not with 10.4.X? Ok maybe I missed something as I never test apps against unstable core (never master, only on tags)

Correct - the change in master makes these tests fail in user_ldap. They started failing in the nightly the night after PR #37398 was merged in core. And there are no other changes in user_ldap code. So it looks like that PR is the source of the problem.

phil-davis · 2020-05-24T09:09:26Z

Latest nightly example: https://drone.owncloud.com/owncloud/user_ldap/2527/48/11

--- Failed scenarios:

    /var/www/owncloud/testrunner/apps/user_ldap/tests/acceptance/features/cliProvisioning/userSync.feature:15
    /var/www/owncloud/testrunner/apps/user_ldap/tests/acceptance/features/cliProvisioning/userSync.feature:25
    /var/www/owncloud/testrunner/apps/user_ldap/tests/acceptance/features/cliProvisioning/userSync.feature:61
    /var/www/owncloud/testrunner/apps/user_ldap/tests/acceptance/features/cliProvisioning/userSync.feature:74
    /var/www/owncloud/testrunner/apps/user_ldap/tests/acceptance/features/cliProvisioning/userSync.feature:87

29 scenarios (24 passed, 5 failed)
210 steps (198 passed, 5 failed, 7 skipped)

phil-davis · 2020-05-24T09:11:01Z

I removed this issue from QA assignment. It seems to be a real problem that needs developer investigation.

mrow4a · 2020-05-24T13:53:57Z

@phil-davis it seems in your testing instance for LDAP, LDAP search for <uid> fails (returns empty result?). I really have trouble reproducing it locally. Tests started failing, because previously there was LDAP search retrieving ALL possible users and filtering inside php, but now it issues search with filter in LDAP and just checks if there is duplicate - https://github.com/owncloud/core/pull/37398/files#diff-0ada4aa7ff5e95845c939f25bf35448dL251-R258 . We can rollback searching for all users, but this is crazy expensive..

I will try to setup that acceptance test suite locally with ldap to develop locally, it is some work.

cc @jvillafanez

phil-davis · 2020-05-24T14:04:29Z

Was this closed accidentally?

mrow4a · 2020-05-24T14:04:48Z

@phil-davis yes, sorry - accident

mrow4a · 2020-05-24T14:38:18Z

@phil-davis @jvillafanez this is possible fix https://github.com/owncloud/core/compare/bugfix/all-users-retr-s-user-sync , but this would be really bad if we need to do that and cannot filter search with LDAP #37428 (comment) on some setups (on mine with https://github.com/owncloud-docker/compose-playground/tree/master/compose/ldap I cannot reproduce it)..

phil-davis · 2020-05-25T03:32:31Z

Added QA labels again - we will try to reproduce manually today, and then see if there is something in the CI-based LDAP configuration that needs to be adjusted to allow single-user searches to the LDAP server.

Based on that, it might be that there are recommendations for LDAP server settings that must be in place for single-user sync to work.

Note: we were also not able to find a way to successfully test the user-sync API that was added recently in PR #36428 - I suspect that this will turn out to be a similar problem. Let's see.

mrow4a · 2020-05-25T07:11:22Z

@phil-davis it can be easy to check - just run ./occ ldap:check-user <uid> to see if you can succesfully filter search user in LDAP server. This should follow similar code as ./occ user:sync -u <uid>.

NOTE:
@phil-davis might be probably important to check how users are provisioned. Note that with LDAP, users need to be added using .ldif files or using ldap commands. Not sure if you can add users through occ (I am not yet aware of code that writes users to LDAP, I know only code that is readonly-mode to interact with LDAP)

jvillafanez · 2020-05-25T07:25:09Z

Not sure if these are the related ldap's logs...

5ec9e923 conn=2268 fd=13 ACCEPT from IP=192.168.128.3:35550 (IP=0.0.0.0:389)
--
12503 | 5ec9e923 conn=2268 op=0 BIND dn="cn=admin,dc=owncloud,dc=com" method=128
12504 | 5ec9e923 conn=2268 op=0 BIND dn="cn=admin,dc=owncloud,dc=com" mech=SIMPLE ssf=0
12505 | 5ec9e923 conn=2268 op=0 RESULT tag=97 err=0 text=
12506 | 5ec9e923 conn=2268 op=1 SRCH base="dc=owncloud,dc=com" scope=2 deref=0 filter="(&(\|(objectClass=inetOrgPerson))(displayName=*)(displayName=user0*))"
12507 | 5ec9e923 conn=2268 op=1 SRCH attr=dn uid samaccountname mail displayName jpegphoto thumbnailphoto
12508 | 5ec9e923 <= mdb_substring_candidates: (displayName) not indexed
12509 | 5ec9e923 conn=2268 op=1 SEARCH RESULT tag=101 err=0 nentries=0 text=
12510 | 5ec9e923 conn=2268 op=2 UNBIND
12511 | 5ec9e923 conn=2268 fd=13 closed


.....
5ec9e93e conn=2339 fd=14 ACCEPT from IP=192.168.128.3:36040 (IP=0.0.0.0:389)
--
13227 | 5ec9e93e conn=2339 op=0 BIND dn="cn=admin,dc=owncloud,dc=com" method=128
13228 | 5ec9e93e conn=2339 op=0 BIND dn="cn=admin,dc=owncloud,dc=com" mech=SIMPLE ssf=0
13229 | 5ec9e93e conn=2339 op=0 RESULT tag=97 err=0 text=
13230 | 5ec9e93e conn=2339 op=1 SRCH base="dc=owncloud,dc=com" scope=2 deref=0 filter="(&(\|(objectClass=inetOrgPerson))(displayName=*)(displayName=user0*))"
13231 | 5ec9e93e conn=2339 op=1 SRCH attr=dn uid samaccountname mail displayName jpegphoto thumbnailphoto
13232 | 5ec9e93e <= mdb_substring_candidates: (displayName) not indexed
13233 | 5ec9e93e conn=2339 op=1 SEARCH RESULT tag=101 err=0 nentries=0 text=
13234 | 5ec9e93e conn=2339 op=2 UNBIND
13235 | 5ec9e93e conn=2339 fd=14 closed

(displayName not indexed shouldn't matter)

In addition,

      │ Deleting accounts:
      │ user0, ,  (no longer exists in the backend)

the user0 doesn't seem to have a display name.

As far as I know, all the user that will be synced in ownCloud MUST have a display name (the displayname=* part of the search filter is fine). However, I'm not sure if that search filter is what we want.

mrow4a · 2020-05-25T08:08:25Z

@jvillafanez $backend->getUsers(<uid>|'', null); should look not only in displayname? This is configurable when setting up the instance, right?

mrow4a · 2020-05-25T09:00:38Z

@jvillafanez @micbar as per discussion with @phil-davis https://github.com/owncloud/user_ldap/blob/master/tests/acceptance/setConfig.sh missing "ldap_attributes_for_user_search": "uid" which probably causes it to search only at displayname (which is also not provided)

jvillafanez · 2020-05-25T09:11:23Z

yes, it seems the main cause is that it only searches using the displayname by default, and you need to explicit set additional attributes.
owncloud/user_ldap#238 seems very related, although it got rejected in the end. It's unlikely that we change things as deep as this one now. We'll have to rely on explicit manual configuration.

mrow4a · 2020-05-25T09:14:06Z

@jvillafanez shall we finish owncloud/user_ldap#238 (comment) ?

close and implement #29503 (comment) That will create new apis, then we can revisit this properly.

jvillafanez · 2020-05-25T09:21:47Z

It's probably too late to add new APIs, but I'm not the one deciding.

phil-davis added dev:acceptance-tests QA:team QA:p1 labels May 21, 2020

phil-davis mentioned this issue May 21, 2020

Use plugins/codecov:latest owncloud/user_ldap#557

Closed

phil-davis self-assigned this May 22, 2020

phil-davis removed their assignment May 24, 2020

phil-davis removed QA:p1 QA:team dev:acceptance-tests labels May 24, 2020

mrow4a closed this as completed May 24, 2020

phil-davis reopened this May 24, 2020

phil-davis added QA:p1 QA:team labels May 25, 2020

kiranparajuli589 self-assigned this May 25, 2020

phil-davis mentioned this issue May 25, 2020

[Tests-Only] Removed unused variable from exception message owncloud/user_ldap#559

Closed

kiranparajuli589 mentioned this issue May 25, 2020

[Tests-Only] Added ldapAttributesForUserSearch in setConfig.sh owncloud/user_ldap#560

Merged

phil-davis closed this as completed in owncloud/user_ldap#560 May 25, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

user_ldap acceptance tests failing #37428

user_ldap acceptance tests failing #37428

phil-davis commented May 21, 2020 •

edited

Loading

phil-davis commented May 22, 2020 •

edited

Loading

mrow4a commented May 22, 2020

phil-davis commented May 22, 2020

phil-davis commented May 24, 2020

phil-davis commented May 24, 2020

mrow4a commented May 24, 2020 •

edited

Loading

phil-davis commented May 24, 2020

mrow4a commented May 24, 2020

mrow4a commented May 24, 2020 •

edited

Loading

phil-davis commented May 25, 2020

mrow4a commented May 25, 2020

jvillafanez commented May 25, 2020

mrow4a commented May 25, 2020

mrow4a commented May 25, 2020

jvillafanez commented May 25, 2020

mrow4a commented May 25, 2020

jvillafanez commented May 25, 2020

user_ldap acceptance tests failing #37428

user_ldap acceptance tests failing #37428

Comments

phil-davis commented May 21, 2020 • edited Loading

phil-davis commented May 22, 2020 • edited Loading

mrow4a commented May 22, 2020

phil-davis commented May 22, 2020

phil-davis commented May 24, 2020

phil-davis commented May 24, 2020

mrow4a commented May 24, 2020 • edited Loading

phil-davis commented May 24, 2020

mrow4a commented May 24, 2020

mrow4a commented May 24, 2020 • edited Loading

phil-davis commented May 25, 2020

mrow4a commented May 25, 2020

jvillafanez commented May 25, 2020

mrow4a commented May 25, 2020

mrow4a commented May 25, 2020

jvillafanez commented May 25, 2020

mrow4a commented May 25, 2020

jvillafanez commented May 25, 2020

phil-davis commented May 21, 2020 •

edited

Loading

phil-davis commented May 22, 2020 •

edited

Loading

mrow4a commented May 24, 2020 •

edited

Loading

mrow4a commented May 24, 2020 •

edited

Loading