vic-ui robot tests #1911

wjun · 2018-08-07T01:45:36Z

Fixes #1771

tmp

This reverts commit c17d914.

This reverts commit f63b61d.

ghost · 2018-08-07T14:35:39Z

LGTM with 1 question above. Great job, Jun!

zjs · 2018-08-07T19:10:31Z

tests/resources/VIC-UI-Util.robot

+
+    :FOR  ${i}  IN RANGE  10
+    \   ${rc}  ${out}=  Run And Return Rc And Output  curl -k -w "\%{http_code}\\n" --header "Content-Type: application/json" -X POST --data '{"vc":{"target":"${vc}:443","user":"${vc_user}","password":"${vc_pass}","thumbprint":"${vc_thumbprint}"},"plugin":{"preset":"${plugin_preset}"}}' https://${ova_ip}:9443/plugin/${action}
+    \   Exit For Loop If  '204' in '''${out}'''


It seems like this (and similar lines) will match 204 appearing anywhere in the curl output (instead of just testing the HTTP status code).

You could use something like this to get the status code by itself:

${out} ${status}= Split String From Right ${out} \n 1

Then, you could check ${status} instead of ${out} (and return ${status} instead of ${out}, so keywords calling this one check the right value too).

Good point. Refactored to get http response status.

zjs · 2018-08-07T19:12:03Z

tests/manual-test-cases/Group9-VIC-UI/9-2-VIC-UI-Uninstaller.robot

+
+*** Keywords *** 
+Uninstall OVA Setup
+    Wait Until Keyword Succeeds  1x  30m  VIC UI OVA Setup


Why do we wait 30 minutes before retrying?

Good catch. I removed this retry.

zjs · 2018-08-07T19:13:26Z

tests/manual-test-cases/Group9-VIC-UI/9-1-VIC-UI-Installer.robot

+*** Settings ***
+Documentation  Test 9-1 - VIC UI Installation
+Resource  ../../resources/Util.robot
+Suite Setup  Wait Until Keyword Succeeds  1x  30m  VIC UI OVA Setup


Why do we wait 30 minutes before retrying?

zjs · 2018-08-07T19:19:23Z

tests/resources/VIC-UI-Util.robot

+Resource  ../resources/Util.robot
+
+*** Keywords ***
+Install UI Plugin


Because this performs two operations, I'm not sure we're validating what we expect in cases where one plugin installation succeeds and one fails. (E.g., tests like Attempt To Install To A Non vCenter Server should probably be verifying that both calls fail, but don't currently do so — and I don't think they can with the current structure.)

Good point. I refactored to add plugin type as a parameter.

zjs · 2018-08-07T19:21:24Z

tests/resources/VIC-UI-Util.robot

+
+    [Return]  ${out}
+
+Deploy OVA And Install UI Plugin And Run Regression Tests


It seems like this keyword is important functionality to test, but I don't see where it's used.

I saw it is used in Group5 test cases.

zjs · 2018-08-07T19:25:30Z

tests/resources/VIC-UI-Util.robot

+    Global Environment Setup
+    Set Test VC Variables
+
+    ${ova-ip}=  Install And Initialize VIC Product OVA  vic-*.ova  %{OVA_NAME}


It seems like vic-*.ova is already repeated in a few places. Perhaps it should be extracted into a variable?

zjs · 2018-08-07T19:26:17Z

tests/resources/VIC-UI-Util.robot

+    Wait Until Keyword Succeeds  10x  5s  Login  root  vmware
+
+    # extract vic bundle name
+    ${download_url}=  Run command and Return output  curl -k https://${ova-ip}:9443 | tac | tac | grep -Po -m 1 '(?<=href=")[^"]*tar.gz'


tac | tac

What's the purpose of this?

wjun · 2018-08-08T03:49:08Z

Why is the TEST_THUMBPRINT using a ‘$’ instead of ‘%’
Jason, I looked at the robot log and found TEST_THUMBPRINT is defined as a robot global environment variable, but implemented as a regular variable, but not a OS environment variable, so I need to use $ to refer to it, while % is for OS environment variable.

zjs · 2018-08-10T17:54:35Z

tests/manual-test-cases/Group9-VIC-UI/9-1-VIC-UI-Installer.robot

@@ -15,29 +15,41 @@
 *** Settings ***
 Documentation  Test 9-1 - VIC UI Installation
 Resource  ../../resources/Util.robot
-Suite Setup  Wait Until Keyword Succeeds  1x  30m  VIC UI OVA Setup
+Suite Setup  Run Keyword  VIC UI OVA Setup


With just a single keyword here, you don't need Run Keyword. You can just use:

Suite Setup VIC UI OVA Setup

zjs · 2018-08-10T17:58:48Z

tests/manual-test-cases/Group9-VIC-UI/9-1-VIC-UI-Installer.robot

 Suite Teardown  Run Keyword And Ignore Error  Nimbus Cleanup  ${list}

 *** Variables ***
 ${ok}=  204
+${html5}=  H5
+${flex}=  FLEX


Since these variables are only intended to be used with Install UI Plugin and Remove UI Plugin, it would probably be best to just define them once in VIC-UI-Util.robot.

* Refactor register tests to use thumbprint tmp * [BROKEN] Robot testing * Refacotr UI plugin registration test cases * Revert "Refacotr UI plugin registration test cases" This reverts commit c17d914. * Revert "Revert "Refacotr UI plugin registration test cases"" This reverts commit f63b61d. * fixup! Refacotr UI plugin registration test cases * fixup! Refacotr UI plugin registration test cases

Refactor register tests to use thumbprint Refacotr UI plugin registration test cases

Jason Morris and others added 5 commits June 6, 2018 17:59

Refactor register tests to use thumbprint

285a3e4

tmp

[BROKEN] Robot testing

1cbecbb

Refacotr UI plugin registration test cases

c17d914

Revert "Refacotr UI plugin registration test cases"

f63b61d

This reverts commit c17d914.

Revert "Revert "Refacotr UI plugin registration test cases""

1e6e3fa

This reverts commit f63b61d.

vmwclabot added the cla-not-required label Aug 7, 2018

wjun requested a review from zjs August 7, 2018 01:50

wjun self-assigned this Aug 7, 2018

wjun requested a review from a user August 7, 2018 01:51

wjun changed the title ~~Morris jason 1771/update tests~~ vic-ui robot tests Aug 7, 2018

ghost approved these changes Aug 7, 2018

View reviewed changes

zjs reviewed Aug 7, 2018

View reviewed changes

fixup! Refacotr UI plugin registration test cases

bb0b083

zjs reviewed Aug 10, 2018

View reviewed changes

This was referenced Aug 10, 2018

Update vic-ui robot tests #1823

Closed

Automated VIC UI Plugin - Nightly/Integration Tests #1770

Closed

fixup! Refacotr UI plugin registration test cases

69d1fb2

wjun merged commit be522d1 into vmware:feature/auto-vic-ui-plugin Aug 16, 2018

wjun added a commit to wjun/vic-product that referenced this pull request Aug 22, 2018

fixup! vic-ui robot tests (vmware#1911)

1b59e25

zjs mentioned this pull request Aug 22, 2018

Merge branch Feature/auto-vic-ui-plugin #1947

Closed

zjs mentioned this pull request Aug 23, 2018

Automated VIC UI Plugin - Updated Tests #1771

Closed

2 tasks

wjun added a commit that referenced this pull request Aug 30, 2018

vic-ui robot tests (#1911)

c8b6805

Refactor register tests to use thumbprint Refacotr UI plugin registration test cases

wjun added a commit that referenced this pull request Aug 30, 2018

vic-ui robot tests (#1911)

100349d

Refactor register tests to use thumbprint Refacotr UI plugin registration test cases

wjun added a commit that referenced this pull request Aug 30, 2018

vic-ui robot tests (#1911)

229358b

Refactor register tests to use thumbprint Refacotr UI plugin registration test cases

wjun added a commit that referenced this pull request Aug 30, 2018

vic-ui robot tests (#1911)

c61a166

Refactor register tests to use thumbprint Refacotr UI plugin registration test cases

wjun added a commit that referenced this pull request Aug 31, 2018

vic-ui robot tests (#1911)

db12a30

Refactor register tests to use thumbprint Refacotr UI plugin registration test cases

wjun added a commit that referenced this pull request Aug 31, 2018

vic-ui robot tests (#1911)

b8aef6a

Refactor register tests to use thumbprint Refacotr UI plugin registration test cases

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

vic-ui robot tests #1911

vic-ui robot tests #1911

wjun commented Aug 7, 2018

ghost commented Aug 7, 2018

zjs Aug 7, 2018 •

edited

Loading

wjun Aug 10, 2018

zjs Aug 7, 2018

wjun Aug 10, 2018

zjs Aug 7, 2018

zjs Aug 7, 2018

wjun Aug 10, 2018

zjs Aug 7, 2018

wjun Aug 10, 2018

zjs Aug 7, 2018

zjs Aug 7, 2018

wjun commented Aug 8, 2018

zjs Aug 10, 2018

wjun Aug 15, 2018

zjs Aug 10, 2018

wjun Aug 15, 2018


		[Return] ${out}

		Deploy OVA And Install UI Plugin And Run Regression Tests

vic-ui robot tests #1911

vic-ui robot tests #1911

Conversation

wjun commented Aug 7, 2018

ghost commented Aug 7, 2018

zjs Aug 7, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wjun commented Aug 8, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zjs Aug 7, 2018 •

edited

Loading