FailedConsole Output

Started by upstream project "doctor-verify-master" build number 152
originally caused by:
 Triggered by Gerrit: https://gerrit.opnfv.org/gerrit/50973
[EnvInject] - Loading node environment variables.
Building remotely on zte-virtual3 (doctor-apex-x86_64) in workspace /home/jenkins/opnfv/slave_root/workspace/doctor-verify-maintenance-apex-sample-x86_64-master
[ssh-agent] Looking for ssh-agent implementation...
[ssh-agent]   Exec ssh-agent (binary ssh-agent on a remote machine)
$ ssh-agent
SSH_AUTH_SOCK=/tmp/ssh-cExU0XpSo0Do/agent.8359
SSH_AGENT_PID=8361
[ssh-agent] Started.
Running ssh-add (command line suppressed)
Identity added: /home/jenkins/opnfv/slave_root/workspace/doctor-verify-maintenance-apex-sample-x86_64-master@tmp/private_key_51698972480288081.key (/home/jenkins/opnfv/slave_root/workspace/doctor-verify-maintenance-apex-sample-x86_64-master@tmp/private_key_51698972480288081.key)
[ssh-agent] Using credentials jenkins-ci (Jenkins Master SSH)
using credential d42411ac011ad6f3dd2e1fa34eaa5d87f910eb2e
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository https://gerrit.opnfv.org/gerrit/doctor
 > git init /home/jenkins/opnfv/slave_root/workspace/doctor-verify-maintenance-apex-sample-x86_64-master # timeout=10
Fetching upstream changes from https://gerrit.opnfv.org/gerrit/doctor
 > git --version # timeout=10
using GIT_SSH to set credentials Jenkins Master SSH
 > git fetch --tags --progress https://gerrit.opnfv.org/gerrit/doctor +refs/heads/*:refs/remotes/origin/* # timeout=15
 > git config remote.origin.url https://gerrit.opnfv.org/gerrit/doctor # timeout=10
 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git config remote.origin.url https://gerrit.opnfv.org/gerrit/doctor # timeout=10
Fetching upstream changes from https://gerrit.opnfv.org/gerrit/doctor
using GIT_SSH to set credentials Jenkins Master SSH
 > git fetch --tags --progress https://gerrit.opnfv.org/gerrit/doctor refs/changes/73/50973/13 # timeout=15
Checking out Revision c942be45f70ec9f1a7dc78a58ba5ebd248eaf4ae (refs/changes/73/50973/13)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f c942be45f70ec9f1a7dc78a58ba5ebd248eaf4ae # timeout=15
Commit message: "[Don't merge] test for CI"
 > git rev-parse FETCH_HEAD^{commit} # timeout=10
 > git rev-list --no-walk 7bba7d67beec7b3db26ab1319bf4080565053096 # timeout=10
No emails were triggered.
[doctor-verify-maintenance-apex-sample-x86_64-master] $ /usr/bin/env bash /tmp/jenkins4504194598315857128.sh
Gathering IP information for Apex installer VM
 1     undercloud                     running
Installer VM detected
Installer ip is 192.168.122.166
fetch_os_creds.info: Fetching rc file...
fetch_os_creds.info: Verifying connectivity to 192.168.122.166...
fetch_os_creds.info: 192.168.122.166 is reachable!
fetch_os_creds.info: ... from Instack VM 192.168.122.166...
Warning: Permanently added '192.168.122.166' (ECDSA) to the list of known hosts.
-------- Credentials: --------
# Clear any old environment that may conflict.
for key in $( set | awk '{FS="="}  /^OS_/ {print $1}' ); do unset $key ; done
export OS_NO_CACHE=True
export COMPUTE_API_VERSION=1.1
export OS_USERNAME=admin
export no_proxy=,192.168.37.20,192.0.2.9
export OS_USER_DOMAIN_NAME=Default
export OS_VOLUME_API_VERSION=3
export OS_CLOUDNAME=overcloud
export OS_AUTH_URL=http://192.168.37.20:5000/v3
export NOVA_VERSION=1.1
export OS_IMAGE_API_VERSION=2
export OS_PASSWORD=Njp1Dn70pcnfZ2Nclp5cp0Uny
export OS_PROJECT_DOMAIN_NAME=Default
export OS_IDENTITY_API_VERSION=3
export OS_PROJECT_NAME=admin
export OS_AUTH_TYPE=password
export PYTHONWARNINGS="ignore:Certificate has no, ignore:A true SSLContext object is not available"

# Add OS_CLOUDNAME to PS1
if [ -z "${CLOUDPROMPT_ENABLED:-}" ]; then
    export PS1=${PS1:-""}
    export PS1=\${OS_CLOUDNAME:+"(\$OS_CLOUDNAME)"}\ $PS1
    export CLOUDPROMPT_ENABLED=1
fi
export OS_PROJECT_ID=4f882fa5f91b48ff840ccef43472dc82
export OS_TENANT_NAME=admin
export OS_REGION_NAME=regionOne
[doctor-verify-maintenance-apex-sample-x86_64-master] $ /bin/sh -xe /tmp/jenkins7702878490492077712.sh
+ source /home/jenkins/opnfv-openrc.sh
+++ set
+++ awk '{FS="="}  /^OS_/ {print $1}'
++ export OS_NO_CACHE=True
++ OS_NO_CACHE=True
++ export COMPUTE_API_VERSION=1.1
++ COMPUTE_API_VERSION=1.1
++ export OS_USERNAME=admin
++ OS_USERNAME=admin
++ export no_proxy=,192.168.37.20,192.0.2.9
++ no_proxy=,192.168.37.20,192.0.2.9
++ export OS_USER_DOMAIN_NAME=Default
++ OS_USER_DOMAIN_NAME=Default
++ export OS_VOLUME_API_VERSION=3
++ OS_VOLUME_API_VERSION=3
++ export OS_CLOUDNAME=overcloud
++ OS_CLOUDNAME=overcloud
++ export OS_AUTH_URL=http://192.168.37.20:5000/v3
++ OS_AUTH_URL=http://192.168.37.20:5000/v3
++ export NOVA_VERSION=1.1
++ NOVA_VERSION=1.1
++ export OS_IMAGE_API_VERSION=2
++ OS_IMAGE_API_VERSION=2
++ export OS_PASSWORD=Njp1Dn70pcnfZ2Nclp5cp0Uny
++ OS_PASSWORD=Njp1Dn70pcnfZ2Nclp5cp0Uny
++ export OS_PROJECT_DOMAIN_NAME=Default
++ OS_PROJECT_DOMAIN_NAME=Default
++ export OS_IDENTITY_API_VERSION=3
++ OS_IDENTITY_API_VERSION=3
++ export OS_PROJECT_NAME=admin
++ OS_PROJECT_NAME=admin
++ export OS_AUTH_TYPE=password
++ OS_AUTH_TYPE=password
++ export 'PYTHONWARNINGS=ignore:Certificate has no, ignore:A true SSLContext object is not available'
++ PYTHONWARNINGS='ignore:Certificate has no, ignore:A true SSLContext object is not available'
++ '[' -z '' ']'
++ export PS1=
++ PS1=
++ export 'PS1=${OS_CLOUDNAME:+($OS_CLOUDNAME)} '
++ PS1='${OS_CLOUDNAME:+($OS_CLOUDNAME)} '
++ export CLOUDPROMPT_ENABLED=1
++ CLOUDPROMPT_ENABLED=1
++ export OS_PROJECT_ID=4f882fa5f91b48ff840ccef43472dc82
++ OS_PROJECT_ID=4f882fa5f91b48ff840ccef43472dc82
++ export OS_TENANT_NAME=admin
++ OS_TENANT_NAME=admin
++ export OS_REGION_NAME=regionOne
++ OS_REGION_NAME=regionOne
+ '[' -f /home/jenkins/os_cacert ']'
+ source /home/jenkins/opnfv-installer.sh
++ export INSTALLER_TYPE=apex
++ INSTALLER_TYPE=apex
++ export INSTALLER_IP=192.168.122.166
++ INSTALLER_IP=192.168.122.166
++ export SSH_KEY=/home/jenkins/installer_key_file
++ SSH_KEY=/home/jenkins/installer_key_file
+ sudo -E tox -e py34
py34 create: /home/jenkins/opnfv/slave_root/workspace/doctor-verify-maintenance-apex-sample-x86_64-master/.tox/py34
py34 installdeps: -r/home/jenkins/opnfv/slave_root/workspace/doctor-verify-maintenance-apex-sample-x86_64-master/requirements.txt
py34 develop-inst: /home/jenkins/opnfv/slave_root/workspace/doctor-verify-maintenance-apex-sample-x86_64-master
py34 installed: You are using pip version 9.0.1, however version 19.0.3 is available.,You should consider upgrading via the 'pip install --upgrade pip' command.,amqp==2.2.1,aodhclient==0.9.0,appdirs==1.4.3,asn1crypto==0.22.0,Babel==2.3.4,bcrypt==3.1.3,cachetools==2.0.0,certifi==2017.4.17,cffi==1.10.0,chardet==3.0.4,click==6.7,cliff==2.8.3,cmd2==0.7.5,contextlib2==0.5.5,cryptography==2.0.2,debtcollector==1.17.2,deprecation==1.0.1,-e git+https://gerrit.opnfv.org/gerrit/doctor@c942be45f70ec9f1a7dc78a58ba5ebd248eaf4ae#egg=doctor_tests,enum-compat==0.0.2,eventlet==0.20.0,fasteners==0.14.1,flake8==2.5.5,Flask==0.12.2,futurist==1.3.2,greenlet==0.4.15,idna==2.5,iso8601==0.1.11,itsdangerous==0.24,Jinja2==2.9.6,jsonpatch==1.16,jsonpointer==1.10,jsonschema==2.6.0,keystoneauth1==3.1.0,kombu==4.1.0,MarkupSafe==1.0,mccabe==0.4.0,monotonic==1.3,msgpack-python==0.4.8,netaddr==0.7.19,netifaces==0.10.6,openstacksdk==0.9.17,os-client-config==1.28.0,osc-lib==1.7.1,oslo.concurrency==3.21.2,oslo.config==4.11.2,oslo.context==2.17.2,oslo.i18n==3.17.1,oslo.log==3.30.3,oslo.messaging==5.30.8,oslo.middleware==3.30.2,oslo.serialization==2.20.2,oslo.service==1.25.2,oslo.utils==3.28.4,oslo.versionedobjects==1.26.3,paramiko==2.2.1,Paste==2.0.3,PasteDeploy==1.5.2,pbr==3.1.1,pep8==1.7.1,pika==0.10.0,pika-pool==0.1.3,positional==1.1.2,prettytable==0.7.2,pyasn1==0.3.1,pycparser==2.18,pyflakes==1.0.0,pyinotify==0.9.6,PyNaCl==1.1.2,pyOpenSSL==17.2.0,pyparsing==2.2.0,pyperclip==1.5.27,python-ceilometerclient==2.9.0,python-cinderclient==3.1.1,python-congressclient==1.8.1,python-dateutil==2.6.1,python-glanceclient==2.8.0,python-heatclient==1.11.1,python-keystoneclient==3.13.0,python-neutronclient==6.5.0,python-novaclient==9.1.2,python-openstackclient==3.12.2,python-swiftclient==3.4.1,python-vitrageclient==1.4.1,pytz==2017.2,PyYAML==3.12,repoze.lru==0.6,requests==2.18.2,requestsexceptions==1.3.0,rfc3986==1.1.0,Routes==2.4.1,scp==0.10.2,simplejson==3.11.1,six==1.10.0,statsd==3.2.1,stevedore==1.25.2,tenacity==4.4.0,urllib3==1.22,vine==1.1.4,virtualenv==15.1.0,warlock==1.2.0,WebOb==1.7.3,Werkzeug==0.12.2,wrapt==1.10.10
py34 runtests: PYTHONHASHSEED='2597557591'
py34 runtests: commands[0] | doctor-test
/home/jenkins/opnfv/slave_root/workspace/doctor-verify-maintenance-apex-sample-x86_64-master/.tox/py34/lib/python3.4/site-packages/paramiko/client.py:711: UserWarning: Unknown ssh-ed25519 host key for 192.168.122.166: b'4374bde45115c808adfe2f4bef95accc'
  key.get_fingerprint())))
2019-04-20 15:48:27,726 main.py 130 INFO   doctor test starting.......
2019-04-20 15:48:27,726 apex.py 43 INFO   Setup Apex installer start......
2019-04-20 15:48:27,727 base.py 113 INFO   Get SSH keys from apex installer......
2019-04-20 15:48:28,125 apex.py 67 INFO   Get overcloud config details from Apex installer......
2019-04-20 15:48:28,126 base.py 174 INFO   Run command=source stackrc; nova list | grep ' overcloud-' in apex installer......
2019-04-20 15:48:34,587 base.py 183 INFO   Output=['| 43583777-fdb1-4b65-8578-f9faaf92fa81 | overcloud-controller-0  | ACTIVE | -          | Running     | ctlplane=192.0.2.20 |', '| ef2ae693-3f97-48ba-9204-4f4648ee46c0 | overcloud-novacompute-0 | ACTIVE | -          | Running     | ctlplane=192.0.2.8  |', '| f8269350-692f-4622-8ebf-d9766f90f9c7 | overcloud-novacompute-1 | ACTIVE | -          | Running     | ctlplane=192.0.2.4  |', '| f35e1511-b513-46f2-ac2f-88fff3fa8cb7 | overcloud-novacompute-2 | ACTIVE | -          | Running     | ctlplane=192.0.2.7  |'] command=source stackrc; nova list | grep ' overcloud-' in apex installer
2019-04-20 15:48:34,588 base.py 188 INFO   Check command=grep docker /home/stack/deploy_command return in apex installer......
2019-04-20 15:48:34,666 base.py 191 INFO   return 0
2019-04-20 15:48:34,666 apex.py 80 INFO   controller_ips:['192.0.2.20']
2019-04-20 15:48:34,667 apex.py 81 INFO   compute_ips:['192.0.2.8', '192.0.2.4', '192.0.2.7']
2019-04-20 15:48:34,667 apex.py 82 INFO   use_containers:True
2019-04-20 15:48:34,667 base.py 174 INFO   Run command=scp overcloudrc heat-admin@192.0.2.20: in apex installer......
2019-04-20 15:48:34,875 main.py 147 ERROR  doctor test failed, Exception=Unable to run 
command: scp overcloudrc heat-admin@192.0.2.20:
ret: 1
2019-04-20 15:48:34,876 main.py 148 ERROR  Traceback (most recent call last):
  File "/home/jenkins/opnfv/slave_root/workspace/doctor-verify-maintenance-apex-sample-x86_64-master/doctor_tests/main.py", line 133, in run
    self.setup()
  File "/home/jenkins/opnfv/slave_root/workspace/doctor-verify-maintenance-apex-sample-x86_64-master/doctor_tests/main.py", line 45, in setup
    self.installer.setup()
  File "/home/jenkins/opnfv/slave_root/workspace/doctor-verify-maintenance-apex-sample-x86_64-master/doctor_tests/installer/apex.py", line 47, in setup
    self._copy_overcloudrc_to_controllers()
  File "/home/jenkins/opnfv/slave_root/workspace/doctor-verify-maintenance-apex-sample-x86_64-master/doctor_tests/installer/apex.py", line 64, in _copy_overcloudrc_to_controllers
    self._run_cmd_remote(self.client, cmd)
  File "/home/jenkins/opnfv/slave_root/workspace/doctor-verify-maintenance-apex-sample-x86_64-master/doctor_tests/installer/base.py", line 176, in _run_cmd_remote
    ret, output = client.ssh(command)
  File "/home/jenkins/opnfv/slave_root/workspace/doctor-verify-maintenance-apex-sample-x86_64-master/doctor_tests/common/utils.py", line 84, in ssh
    % (command, ret))
Exception: Unable to run 
command: scp overcloudrc heat-admin@192.0.2.20:
ret: 1

2019-04-20 15:48:34,877 apex.py 166 INFO   restore apply patches start......
/home/jenkins/opnfv/slave_root/workspace/doctor-verify-maintenance-apex-sample-x86_64-master/.tox/py34/lib/python3.4/site-packages/paramiko/client.py:711: UserWarning: Unknown ssh-ed25519 host key for 192.0.2.20: b'0faab8c75a4be1124e7a573059eb2937'
  key.get_fingerprint())))
2019-04-20 15:48:35,013 image.py 71 INFO   image delete start.......
2019-04-20 15:48:35,014 image.py 76 INFO   image delete end.......
2019-04-20 15:48:35,015 user.py 163 INFO   user delete start......
2019-04-20 15:48:35,015 user.py 187 INFO   user delete end......
2019-04-20 15:48:35,929 base.py 218 INFO   Command sudo python restore_compute_config.py output ['nova.bak does not exist.']
2019-04-20 15:48:35,938 base.py 218 INFO   Command sudo python restore_compute_config.py output ['nova.bak does not exist.']
2019-04-20 15:48:35,941 base.py 218 INFO   Command sudo python restore_config.py output ['Bak_file:/var/lib/config-data/puppet-generated/ceilometer/etc/ceilometer/event_pipeline.yaml.bak does not exist.', "Bak_file doesn't exist: /var/lib/config-data/puppet-generated/ceilometer/etc/ceilometer/event_definitions.bak."]
2019-04-20 15:48:35,941 base.py 218 INFO   Command sudo python restore_compute_config.py output ['nova.bak does not exist.']
2019-04-20 15:48:36,154 base.py 218 INFO   Command sudo python restore_aodh.py output []
2019-04-20 15:48:36,311 base.py 218 INFO   Command sudo python restore_compute_config.py output ['nova.bak does not exist.']
ERROR: InvocationError for command '/home/jenkins/opnfv/slave_root/workspace/doctor-verify-maintenance-apex-sample-x86_64-master/.tox/py34/bin/doctor-test' (exited with code 1)
___________________________________ summary ____________________________________
ERROR:   py34: commands failed
Build step 'Execute shell' marked build as failure
$ ssh-agent -k
unset SSH_AUTH_SOCK;
unset SSH_AGENT_PID;
echo Agent pid 8361 killed;
[ssh-agent] Stopped.
Archiving artifacts
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Request made to compress build log
Sending email to: tbramwell@linuxfoundation.org agardner@linuxfoundation.org rgrigar@linuxfoundation.org
[WS-CLEANUP] Deleting project workspace...
[WS-CLEANUP] Deferred wipeout is used...
[WS-CLEANUP] done
Finished: FAILURE