FailedConsole Output

Skipping 92 KB.. Full Log
ed: 17:07:22.943733
    Duration: 33.937 ms
     Changes:   
----------
          ID: salt_update_certificates
    Function: cmd.wait
        Name: update-ca-certificates
      Result: True
     Comment: 
     Started: 17:07:22.979506
    Duration: 0.877 ms
     Changes:   
----------
          ID: salt_minion_service_restart
    Function: cmd.run
        Name: while true; do salt-call saltutil.running|grep fun: && continue; salt-call --local service.restart salt-minion; break; done
      Result: True
     Comment: Command "while true; do salt-call saltutil.running|grep fun: && continue; salt-call --local service.restart salt-minion; break; done" run
     Started: 17:07:22.983764
    Duration: 52.84 ms
     Changes:   
              ----------
              pid:
                  2627
              retcode:
                  None
              stderr:
              stdout:
                  None

Summary for local
-------------
Succeeded: 66 (changed=54)
Failed:     0
-------------
Total states run:     66
Total run time:   15.213 s
[INFO    ] Could not determine init system from command line: (/bin/tini -- /entrypoint.sh)
[INFO    ] Could not determine init system from command line: (/bin/tini -- /entrypoint.sh)
[INFO    ] Executing command ['service', 'salt-minion', 'restart'] in directory '/home/ubuntu'
local:
    True
[wait_for] OK: salt-call state.sls reclass,linux.network,salt.minion               exclude='[{id: reclass_packages}, {id: /etc/reclass/reclass-config.yml}]'

[wait_for] Waiting for cmd to return success: salt -C 'E@^(mas01).*' saltutil.refresh_pillar
[wait_for] Attempt 1/3 for: salt -C 'E@^(mas01).*' saltutil.refresh_pillar
mas01.mcp-odl-noha.local:
    True
[wait_for] OK: salt -C 'E@^(mas01).*' saltutil.refresh_pillar

[wait_for] Waiting for cmd to return success: (for n in mas01; do salt -C ${n}.* test.ping || exit; done)
[wait_for] Attempt 1/3 for: (for n in mas01; do salt -C ${n}.* test.ping || exit; done)
mas01.mcp-odl-noha.local:
    True
[wait_for] OK: (for n in mas01; do salt -C ${n}.* test.ping || exit; done)

[wait_for] Waiting for cmd to return success: salt -C 'E@^(mas01).*' saltutil.sync_all
[wait_for] Attempt 1/3 for: salt -C 'E@^(mas01).*' saltutil.sync_all
mas01.mcp-odl-noha.local:
    ----------
    beacons:
    clouds:
    engines:
        - engines.architect
        - engines.saltgraph
    grains:
        - grains.ceilometer_policy
        - grains.ceph
        - grains.cinder_policy
        - grains.docker_swarm
        - grains.glance_policy
        - grains.heat_policy
        - grains.jenkins_plugins
        - grains.keystone_policy
        - grains.kubernetes
        - grains.neutron_policy
        - grains.nova_policy
        - grains.ssh_fingerprints
    log_handlers:
    modules:
        - modules._modules.gnocchiv1.__init__
        - modules._modules.gnocchiv1.archive_policy
        - modules._modules.gnocchiv1.common
        - modules.architect
        - modules.artifactory
        - modules.auditd
        - modules.avinetworks
        - modules.barbicanv1.__init__
        - modules.barbicanv1.acl
        - modules.barbicanv1.common
        - modules.barbicanv1.secrets
        - modules.ceph_ng
        - modules.cfgdrive
        - modules.cinderng
        - modules.cinderv3.__init__
        - modules.cinderv3.arg_converter
        - modules.cinderv3.common
        - modules.cinderv3.lists
        - modules.cinderv3.services
        - modules.cinderv3.volume
        - modules.cinderv3.volume_actions
        - modules.cinderv3.volume_types
        - modules.cinderv3.volumes
        - modules.configdrive
        - modules.contrail
        - modules.contrail_health
        - modules.creds
        - modules.devops_utils
        - modules.dockerng_service
        - modules.encode_json
        - modules.gerrit
        - modules.gitlab
        - modules.glanceng
        - modules.glancev2.__init__
        - modules.glancev2.common
        - modules.glancev2.image
        - modules.glancev2.task
        - modules.gnocchiv1.__init__
        - modules.gnocchiv1.archive_policy
        - modules.gnocchiv1.common
        - modules.gnocchiv1.gnocchiv1.__init__
        - modules.gnocchiv1.gnocchiv1.archive_policy
        - modules.gnocchiv1.gnocchiv1.common
        - modules.heat
        - modules.heatv1.__init__
        - modules.heatv1.common
        - modules.heatv1.services
        - modules.heatv1.stack
        - modules.heka_alarming
        - modules.helm
        - modules.iptables_extra
        - modules.ironicng
        - modules.ironicv1.__init__
        - modules.ironicv1.chassis
        - modules.ironicv1.common
        - modules.ironicv1.drivers
        - modules.ironicv1.nodes
        - modules.ironicv1.ports
        - modules.ironicv1.volumes
        - modules.jenkins_common
        - modules.keystone_policy
        - modules.keystoneng
        - modules.keystonev3.__init__
        - modules.keystonev3.arg_converter
        - modules.keystonev3.common
        - modules.keystonev3.domains
        - modules.keystonev3.endpoints
        - modules.keystonev3.groups
        - modules.keystonev3.lists
        - modules.keystonev3.projects
        - modules.keystonev3.regions
        - modules.keystonev3.roles
        - modules.keystonev3.services
        - modules.keystonev3.users
        - modules.linux_hosts
        - modules.linux_netlink
        - modules.maas
        - modules.maas_client
        - modules.maasng
        - modules.manilang.__init__
        - modules.manilang.common
        - modules.manilang.share_types
        - modules.manilang.shares
        - modules.modelschema
        - modules.modelutils
        - modules.multipart
        - modules.nagios_alarming
        - modules.net_checks
        - modules.netutils
        - modules.neutronng
        - modules.neutronv2.__init__
        - modules.neutronv2.agents
        - modules.neutronv2.arg_converter
        - modules.neutronv2.auto_alloc
        - modules.neutronv2.common
        - modules.neutronv2.lists
        - modules.neutronv2.networks
        - modules.neutronv2.ports
        - modules.neutronv2.rbac
        - modules.neutronv2.routers
        - modules.neutronv2.subnetpools
        - modules.neutronv2.subnets
        - modules.novang
        - modules.novav21.__init__
        - modules.novav21.aggregates
        - modules.novav21.common
        - modules.novav21.flavors
        - modules.novav21.keypairs
        - modules.novav21.quotas
        - modules.novav21.servers
        - modules.novav21.services
        - modules.octaviav2.__init__
        - modules.octaviav2.common
        - modules.octaviav2.loadbalancers
        - modules.ovs_config
        - modules.reclass
        - modules.rsyslog_util
        - modules.rundeck
        - modules.saltkey
        - modules.saltresource
        - modules.seedng
        - modules.testing.__init__
        - modules.testing.credentials
        - modules.testing.django
        - modules.testing.django_client_proxy
        - modules.utils
        - modules.virtng
    output:
    proxymodules:
    renderers:
    returners:
        - returners.postgres_graph_db
    sdb:
    states:
        - states._states.gnocchiv1
        - states.artifactory
        - states.avinetworks
        - states.barbicanv1
        - states.cinderng
        - states.cinderv3
        - states.contrail
        - states.contrail_health
        - states.debmirror
        - states.dockerng_service
        - states.gerrit
        - states.gitlab
        - states.glanceng
        - states.glancev2
        - states.gnocchiv1
        - states.grafana3_dashboard
        - states.grafana3_datasource
        - states.heat
        - states.heatv1
        - states.helm_release
        - states.helm_repos
        - states.httpng
        - states.ironicng
        - states.ironicv1
        - states.jenkins_approval
        - states.jenkins_artifactory
        - states.jenkins_credential
        - states.jenkins_gerrit
        - states.jenkins_globalenvprop
        - states.jenkins_jira
        - states.jenkins_job
        - states.jenkins_lib
        - states.jenkins_node
        - states.jenkins_plugin
        - states.jenkins_security
        - states.jenkins_slack
        - states.jenkins_smtp
        - states.jenkins_theme
        - states.jenkins_throttle_category
        - states.jenkins_user
        - states.jenkins_view
        - states.keystone_policy
        - states.keystoneng
        - states.keystonev3
        - states.kibana_object
        - states.maasng
        - states.manilang
        - states.neutronng
        - states.neutronv2
        - states.novang
        - states.novav21
        - states.octaviav2
        - states.ovs_config
        - states.powerdns_mysql
        - states.reclass
        - states.rundeck_project
        - states.rundeck_scm
        - states.rundeck_secret
    utils:
[wait_for] OK: salt -C 'E@^(mas01).*' saltutil.sync_all
[wait_for] OK: ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i /var/lib/opnfv/mcp.rsa ubuntu@10.20.0.2 sudo             CI_DEBUG=false ERASE_ENV=0             /root/fuel/mcp/config/states/virtual_init
[STATE] Applying state: maas
(B
[wait_for] Waiting for cmd to return success: ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i /var/lib/opnfv/mcp.rsa ubuntu@10.20.0.2 sudo             CI_DEBUG=false ERASE_ENV=0             /root/fuel/mcp/config/states/maas
[wait_for] Attempt 1/5 for: ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i /var/lib/opnfv/mcp.rsa ubuntu@10.20.0.2 sudo             CI_DEBUG=false ERASE_ENV=0             /root/fuel/mcp/config/states/maas
Warning: Permanently added '10.20.0.2' (ECDSA) to the list of known hosts.

[wait_for] Waiting for cmd to return success: salt --out yaml -C 'mas01*' service.status maas-fixup | fgrep -q 'false'
[wait_for] Attempt 1/60 for: salt --out yaml -C 'mas01*' service.status maas-fixup | fgrep -q 'false'
No response
[wait_for] Attempt 2/60 for: salt --out yaml -C 'mas01*' service.status maas-fixup | fgrep -q 'false'
No response
[wait_for] Attempt 3/60 for: salt --out yaml -C 'mas01*' service.status maas-fixup | fgrep -q 'false'
No response
[wait_for] Attempt 4/60 for: salt --out yaml -C 'mas01*' service.status maas-fixup | fgrep -q 'false'
No response
[wait_for] Attempt 5/60 for: salt --out yaml -C 'mas01*' service.status maas-fixup | fgrep -q 'false'
No response
[wait_for] Attempt 6/60 for: salt --out yaml -C 'mas01*' service.status maas-fixup | fgrep -q 'false'
No response
[wait_for] Attempt 7/60 for: salt --out yaml -C 'mas01*' service.status maas-fixup | fgrep -q 'false'
No response
[wait_for] Attempt 8/60 for: salt --out yaml -C 'mas01*' service.status maas-fixup | fgrep -q 'false'
No response
[wait_for] Attempt 9/60 for: salt --out yaml -C 'mas01*' service.status maas-fixup | fgrep -q 'false'
No response
[wait_for] Attempt 10/60 for: salt --out yaml -C 'mas01*' service.status maas-fixup | fgrep -q 'false'
[wait_for] OK: salt --out yaml -C 'mas01*' service.status maas-fixup | fgrep -q 'false'

[wait_for] Waiting for cmd to return success: salt -C 'mas01*' state.apply salt,iptables
[wait_for] Attempt 1/10 for: salt -C 'mas01*' state.apply salt,iptables
mas01.mcp-odl-noha.local:
----------
          ID: salt_minion_packages
    Function: pkg.installed
        Name: salt-minion
      Result: True
     Comment: All specified packages are already installed
     Started: 17:11:10.238164
    Duration: 313.521 ms
     Changes:   
----------
          ID: salt_minion_dependency_packages
    Function: pkg.installed
      Result: True
     Comment: All specified packages are already installed
     Started: 17:11:10.552047
    Duration: 18.039 ms
     Changes:   
----------
          ID: /etc/salt/minion.d/minion.conf
    Function: file.managed
      Result: True
     Comment: File /etc/salt/minion.d/minion.conf updated
     Started: 17:11:10.622415
    Duration: 1069.745 ms
     Changes:   
              ----------
              diff:
                  New file
----------
          ID: salt_minion_systemd_override
    Function: file.managed
        Name: /etc/systemd/system/salt-minion.service.d/50-restarts.conf
      Result: True
     Comment: File /etc/systemd/system/salt-minion.service.d/50-restarts.conf updated
     Started: 17:11:11.705791
    Duration: 104.428 ms
     Changes:   
              ----------
              diff:
                  New file
              mode:
                  0644
----------
          ID: salt_minion_systemd_reload
    Function: module.wait
        Name: service.systemctl_reload
      Result: True
     Comment: 
     Started: 17:11:11.810802
    Duration: 0.57 ms
     Changes:   
----------
          ID: salt_minion_service
    Function: service.running
        Name: salt-minion
      Result: True
     Comment: The service salt-minion is already running
     Started: 17:11:11.812214
    Duration: 565.629 ms
     Changes:   
----------
          ID: salt_minion_sync_all
    Function: module.run
        Name: saltutil.sync_all
      Result: True
     Comment: State was not run because none of the onchanges reqs changed
     Started: 17:11:12.379575
    Duration: 0.042 ms
     Changes:   
----------
          ID: salt_minion_grains_dir
    Function: file.directory
        Name: /etc/salt/grains.d
      Result: True
     Comment: Directory /etc/salt/grains.d updated
     Started: 17:11:12.379969
    Duration: 2.655 ms
     Changes:   
              ----------
              /etc/salt/grains.d:
                  New Dir
----------
          ID: salt_minion_grains_files
    Function: file.managed
        Name: /etc/salt/grains
      Result: True
     Comment: Empty file
     Started: 17:11:12.383575
    Duration: 1.679 ms
     Changes:   
              ----------
              new:
                  file /etc/salt/grains created
----------
          ID: salt_minion_grains_files
    Function: file.managed
        Name: /etc/salt/grains.d/placeholder
      Result: True
     Comment: Empty file
     Started: 17:11:12.385801
    Duration: 1.335 ms
     Changes:   
              ----------
              new:
                  file /etc/salt/grains.d/placeholder created
----------
          ID: salt_minion_grain_linux_sphinx
    Function: file.managed
        Name: /etc/salt/grains.d/sphinx
      Result: True
     Comment: File /etc/salt/grains.d/sphinx updated
     Started: 17:11:12.387689
    Duration: 2.834 ms
     Changes:   
              ----------
              diff:
                  New file
----------
          ID: salt_minion_grain_linux_sphinx_validity_check
    Function: cmd.wait
        Name: python -c "import yaml; stream = file('/etc/salt/grains.d/sphinx', 'r'); yaml.load(stream); stream.close()"
      Result: True
     Comment: Command "python -c "import yaml; stream = file('/etc/salt/grains.d/sphinx', 'r'); yaml.load(stream); stream.close()"" run
     Started: 17:11:12.405158
    Duration: 639.848 ms
     Changes:   
              ----------
              pid:
                  3346
              retcode:
                  0
              stderr:
              stdout:
----------
          ID: salt_minion_grain_linux_dns_records
    Function: file.managed
        Name: /etc/salt/grains.d/dns_records
      Result: True
     Comment: File /etc/salt/grains.d/dns_records updated
     Started: 17:11:13.045942
    Duration: 8.696 ms
     Changes:   
              ----------
              diff:
                  New file
----------
          ID: salt_minion_grain_linux_dns_records_validity_check
    Function: cmd.wait
        Name: python -c "import yaml; stream = file('/etc/salt/grains.d/dns_records', 'r'); yaml.load(stream); stream.close()"
      Result: True
     Comment: Command "python -c "import yaml; stream = file('/etc/salt/grains.d/dns_records', 'r'); yaml.load(stream); stream.close()"" run
     Started: 17:11:13.056946
    Duration: 428.965 ms
     Changes:   
              ----------
              pid:
                  3350
              retcode:
                  0
              stderr:
              stdout:
----------
          ID: salt_minion_grain_salt_salt
    Function: file.managed
        Name: /etc/salt/grains.d/salt
      Result: True
     Comment: File /etc/salt/grains.d/salt updated
     Started: 17:11:13.486828
    Duration: 14.763 ms
     Changes:   
              ----------
              diff:
                  New file
----------
          ID: salt_minion_grain_salt_salt_validity_check
    Function: cmd.wait
        Name: python -c "import yaml; stream = file('/etc/salt/grains.d/salt', 'r'); yaml.load(stream); stream.close()"
      Result: True
     Comment: Command "python -c "import yaml; stream = file('/etc/salt/grains.d/salt', 'r'); yaml.load(stream); stream.close()"" run
     Started: 17:11:13.503799
    Duration: 243.177 ms
     Changes:   
              ----------
              pid:
                  3352
              retcode:
                  0
              stderr:
              stdout:
----------
          ID: salt_minion_grains_file
    Function: cmd.wait
        Name: cat /etc/salt/grains.d/* > /etc/salt/grains
      Result: True
     Comment: Command "cat /etc/salt/grains.d/* > /etc/salt/grains" run
     Started: 17:11:13.751611
    Duration: 86.398 ms
     Changes:   
              ----------
              pid:
                  3354
              retcode:
                  0
              stderr:
              stdout:
----------
          ID: salt_minion_grains_publish
    Function: module.wait
        Name: mine.update
      Result: True
     Comment: Module function mine.update executed
     Started: 17:11:13.840579
    Duration: 2.577 ms
     Changes:   
              ----------
              ret:
                  None
----------
          ID: salt_ca_certificates_packages
    Function: pkg.installed
        Name: ca-certificates
      Result: True
     Comment: All specified packages are already installed
     Started: 17:11:13.843376
    Duration: 15.156 ms
     Changes:   
----------
          ID: salt_update_certificates
    Function: cmd.wait
        Name: update-ca-certificates
      Result: True
     Comment: 
     Started: 17:11:13.859587
    Duration: 0.734 ms
     Changes:   
----------
          ID: iptables_packages_v4
    Function: pkg.installed
        Name: iptables
      Result: True
     Comment: All specified packages are already installed
     Started: 17:11:13.860564
    Duration: 12.421 ms
     Changes:   
----------
          ID: iptables_packages_v4
    Function: pkg.installed
        Name: iptables-persistent
      Result: True
     Comment: All specified packages are already installed
     Started: 17:11:13.873253
    Duration: 21.281 ms
     Changes:   
----------
          ID: iptables_modules_v4_load
    Function: kmod.present
      Result: True
     Comment: Loaded kernel modules iptable_filter, ip_tables
     Started: 17:11:13.909517
    Duration: 730.265 ms
     Changes:   
              ----------
              ip_tables:
                  loaded
              iptable_filter:
                  loaded
----------
          ID: /etc/iptables/rules.v4
    Function: file.managed
      Result: True
     Comment: File /etc/iptables/rules.v4 updated
     Started: 17:11:14.640717
    Duration: 573.341 ms
     Changes:   
              ----------
              diff:
                  New file
              mode:
                  0640
----------
          ID: iptables_services_v4_start
    Function: cmd.run
        Name: find /usr/share/netfilter-persistent/plugins.d/[0-9]*-ip4tables -exec {} start \;
      Result: True
     Comment: onlyif execution failed
     Started: 17:11:15.215553
    Duration: 161.836 ms
     Changes:   
----------
          ID: netfilter-persistent
    Function: service.running
      Result: True
     Comment: Service restarted
     Started: 17:11:15.590205
    Duration: 418.801 ms
     Changes:   
              ----------
              netfilter-persistent:
                  True
----------
          ID: iptables_tables_cleanup_v4
    Function: module.wait
        Name: iptables_extra.remove_stale_tables
      Result: True
     Comment: Module function iptables_extra.remove_stale_tables executed
     Started: 17:11:16.011085
    Duration: 58.202 ms
     Changes:   
              ----------
              ret:
                  None
----------
          ID: iptables_services_v6_stop
    Function: cmd.run
        Name: find /usr/share/netfilter-persistent/plugins.d/[0-9]*-ip6tables -exec {} flush \;
      Result: True
     Comment: onlyif execution failed
     Started: 17:11:16.069735
    Duration: 32.488 ms
     Changes:   
----------
          ID: /etc/iptables/rules.v6
    Function: file.absent
      Result: True
     Comment: File /etc/iptables/rules.v6 is not present
     Started: 17:11:16.104254
    Duration: 1.221 ms
     Changes:   
----------
          ID: iptables_tables_flush_v6
    Function: module.wait
        Name: iptables_extra.flush_all
      Result: True
     Comment: 
     Started: 17:11:16.106434
    Duration: 0.635 ms
     Changes:   
----------
          ID: salt_minion_service_restart
    Function: cmd.run
        Name: while true; do salt-call saltutil.running|grep fun: && continue; salt-call --local service.restart salt-minion; break; done
      Result: True
     Comment: Command "while true; do salt-call saltutil.running|grep fun: && continue; salt-call --local service.restart salt-minion; break; done" run
     Started: 17:11:16.124782
    Duration: 36.332 ms
     Changes:   
              ----------
              pid:
                  3387
              retcode:
                  None
              stderr:
              stdout:
                  None

Summary for mas01.mcp-odl-noha.local
-------------
Succeeded: 31 (changed=18)
Failed:     0
-------------
Total states run:     31
Total run time:    5.568 s
[wait_for] OK: salt -C 'mas01*' state.apply salt,iptables
mas01.mcp-odl-noha.local:
----------
          ID: maas_cluster_packages
    Function: pkg.installed
        Name: maas-rack-controller
      Result: True
     Comment: All specified packages are already installed
     Started: 17:11:32.355928
    Duration: 222.857 ms
     Changes:   
----------
          ID: maas_cluster_packages
    Function: pkg.installed
        Name: ipmitool
      Result: True
     Comment: All specified packages are already installed
     Started: 17:11:32.579179
    Duration: 26.227 ms
     Changes:   
----------
          ID: /etc/maas/rackd.conf
    Function: file.line
      Result: True
     Comment: Changes were made
     Started: 17:11:32.612111
    Duration: 10.083 ms
     Changes:   
              ----------
              diff:
                  --- 
                  +++ 
                  @@ -1,2 +1,2 @@
                   cluster_uuid: d5154d6f-523e-4e67-8d72-f8723f98e15d
                  -maas_url: http://localhost:5240/MAAS
                  +maas_url: http://10.1.0.3:5240/MAAS
----------
          ID: /etc/maas/rackd_conf
    Function: file.managed
        Name: /etc/maas/rackd.conf
      Result: True
     Comment: File /etc/maas/rackd.conf exists with proper permissions. No changes made.
     Started: 17:11:32.622475
    Duration: 2.141 ms
     Changes:   
----------
          ID: maas_cluster_services
    Function: service.running
        Name: maas-rackd
      Result: True
     Comment: Service restarted
     Started: 17:11:32.873199
    Duration: 1826.537 ms
     Changes:   
              ----------
              maas-rackd:
                  True

Summary for mas01.mcp-odl-noha.local
------------
Succeeded: 5 (changed=2)
Failed:    0
------------
Total states run:     5
Total run time:   2.088 s

[wait_for] Waiting for cmd to return success: salt -C 'mas01*' state.apply maas.region
[wait_for] Attempt 1/10 for: salt -C 'mas01*' state.apply maas.region
mas01.mcp-odl-noha.local:
----------
          ID: maas_region_packages
    Function: pkg.installed
        Name: maas-region-controller
      Result: True
     Comment: All specified packages are already installed
     Started: 17:11:42.136430
    Duration: 333.597 ms
     Changes:   
----------
          ID: maas_region_packages
    Function: pkg.installed
        Name: python-oauth
      Result: True
     Comment: All specified packages are already installed
     Started: 17:11:42.470413
    Duration: 89.436 ms
     Changes:   
----------
          ID: /etc/maas/regiond.conf
    Function: file.replace
      Result: True
     Comment: Changes were made
     Started: 17:11:42.565915
    Duration: 48.756 ms
     Changes:   
              ----------
              diff:
                  --- 
                  +++ 
                  @@ -3,4 +3,4 @@
                   database_pass: cxSwm3l27k0o
                   database_port: 5432
                   database_user: maas
                  -maas_url: http://10.20.0.3:5240/MAAS
                  +maas_url: http://10.1.0.3:5240/MAAS
----------
          ID: /usr/lib/python3/dist-packages/provisioningserver/templates/proxy/maas-proxy.conf.template
    Function: file.managed
      Result: True
     Comment: File /usr/lib/python3/dist-packages/provisioningserver/templates/proxy/maas-proxy.conf.template updated
     Started: 17:11:42.615264
    Duration: 475.472 ms
     Changes:   
              ----------
              diff:
                  --- 
                  +++ 
                  @@ -1,3 +1,5 @@
                  +
                  +
                   # DO NOT EDIT.  This file is automatically created by MAAS.
                   # Last updated at {{modified}}.
                   
                  @@ -10,6 +12,7 @@
                   acl localnet src {{cidr}}
                   {{endfor}}
                   acl SSL_ports port 443
                  +acl SSL_ports port 50000        # jenkins slave
                   acl Safe_ports port 80          # http
                   acl Safe_ports port 21          # ftp
                   acl Safe_ports port 443         # https
                  @@ -23,11 +26,7 @@
                   http_access allow localhost
                   http_access deny all
                   http_port 3128 transparent
                  -{{if not maas_proxy_port}}
                   http_port 8000
                  -{{else}}
                  -http_port {{maas_proxy_port}}
                  -{{endif}}
                   refresh_pattern ^ftp:           1440    20%     10080
                   refresh_pattern -i (/cgi-bin/|\?) 0     0%      0
                   refresh_pattern \/Release(|\.gpg)$                        0    0%    0 refresh-ims
                  @@ -61,7 +60,5 @@
                   cache_log /var/log/maas/proxy/cache.log
                   cache_store_log /var/log/maas/proxy/store.log
                   {{endif}}
                  -{{if upstream_peer_proxy}}
                  -cache_peer {{upstream_proxy_address}} parent {{upstream_proxy_port}} 0 no-query default
                  -never_direct allow all
                  -{{endif}}
                  +
                  +
----------
          ID: maas_timeout_commissioning
    Function: file.replace
        Name: /usr/lib/python3/dist-packages/maasserver/node_status.py
      Result: True
     Comment: Changes were made
     Started: 17:11:43.091337
    Duration: 10.567 ms
     Changes:   
              ----------
              diff:
                  --- 
                  +++ 
                  @@ -284,7 +284,7 @@
                   }
                   
                   NODE_FAILURE_MONITORED_STATUS_TIMEOUTS = {
                  -    NODE_STATUS.COMMISSIONING: 20,
                  +    NODE_STATUS.COMMISSIONING: 10,
                       NODE_STATUS.DEPLOYING: 40,
                       NODE_STATUS.RELEASING: 5,
                       NODE_STATUS.ENTERING_RESCUE_MODE: 20,
----------
          ID: maas_timeout_deploying
    Function: file.replace
        Name: /usr/lib/python3/dist-packages/maasserver/node_status.py
      Result: True
     Comment: Changes were made
     Started: 17:11:43.102491
    Duration: 10.308 ms
     Changes:   
              ----------
              diff:
                  --- 
                  +++ 
                  @@ -285,7 +285,7 @@
                   
                   NODE_FAILURE_MONITORED_STATUS_TIMEOUTS = {
                       NODE_STATUS.COMMISSIONING: 10,
                  -    NODE_STATUS.DEPLOYING: 40,
                  +    NODE_STATUS.DEPLOYING: 15,
                       NODE_STATUS.RELEASING: 5,
                       NODE_STATUS.ENTERING_RESCUE_MODE: 20,
                       NODE_STATUS.EXITING_RESCUE_MODE: 5,
----------
          ID: maas_interface_default_mode_dhcp
    Function: file.replace
        Name: /usr/lib/python3/dist-packages/maasserver/models/node.py
      Result: True
     Comment: Changes were made
     Started: 17:11:43.113433
    Duration: 134.1 ms
     Changes:   
              ----------
              diff:
                  --- 
                  +++ 
                  @@ -3210,7 +3210,7 @@
                               alloc_type=IPADDRESS_TYPE.DISCOVERED, subnet__isnull=False)
                           for ip_address in discovered_addresses:
                               boot_interface.link_subnet(
                  -                INTERFACE_LINK_TYPE.AUTO, ip_address.subnet)
                  +                INTERFACE_LINK_TYPE.DHCP, ip_address.subnet)
                               auto_set = True
                           if not auto_set:
                               # Failed to set AUTO mode on the boot interface. Lets force an
----------
          ID: /etc/apache2/conf-enabled/maas-http.conf
    Function: file.managed
      Result: True
     Comment: File /etc/apache2/conf-enabled/maas-http.conf updated
     Started: 17:11:43.248224
    Duration: 194.309 ms
     Changes:   
              ----------
              diff:
                  --- 
                  +++ 
                  @@ -37,3 +37,5 @@
                       # Redirect (permanently) requests for /MAAS to /MAAS/.
                       RewriteRule ^/MAAS$ %{REQUEST_URI}/ [R=301,L]
                   </IfModule>
                  +Header set Access-Control-Allow-Origin "*"
                  +Header set X-Frame-Options "ALLOWALL"
----------
          ID: maas_apache_headers
    Function: cmd.run
        Name: a2enmod headers
      Result: True
     Comment: Command "a2enmod headers" run
     Started: 17:11:43.444857
    Duration: 316.097 ms
     Changes:   
              ----------
              pid:
                  3728
              retcode:
                  0
              stderr:
              stdout:
                  Enabling module headers.
                  To activate the new configuration, you need to run:
                    service apache2 restart
----------
          ID: /usr/share/maas/web/static/css/maas-styles.css
    Function: file.managed
      Result: True
     Comment: File /usr/share/maas/web/static/css/maas-styles.css updated
     Started: 17:11:43.761344
    Duration: 160.328 ms
     Changes:   
              ----------
              diff:
                  New file
              mode:
                  0644
----------
          ID: /etc/maas/preseeds/curtin_userdata_amd64_generic_trusty
    Function: file.managed
      Result: True
     Comment: File /etc/maas/preseeds/curtin_userdata_amd64_generic_trusty updated
     Started: 17:11:43.922350
    Duration: 207.016 ms
     Changes:   
              ----------
              diff:
                  New file
              mode:
                  0644
----------
          ID: /etc/maas/preseeds/curtin_userdata_amd64_generic_xenial
    Function: file.managed
      Result: True
     Comment: File /etc/maas/preseeds/curtin_userdata_amd64_generic_xenial updated
     Started: 17:11:44.129998
    Duration: 257.163 ms
     Changes:   
              ----------
              diff:
                  New file
              mode:
                  0644
----------
          ID: /etc/maas/preseeds/curtin_userdata_arm64_generic_xenial
    Function: file.managed
      Result: True
     Comment: File /etc/maas/preseeds/curtin_userdata_arm64_generic_xenial updated
     Started: 17:11:44.387828
    Duration: 337.666 ms
     Changes:   
              ----------
              diff:
                  New file
              mode:
                  0644
----------
          ID: Configure /root/.pgpass for MAAS
    Function: file.managed
        Name: /root/.pgpass
      Result: True
     Comment: File /root/.pgpass updated
     Started: 17:11:44.725780
    Duration: 244.664 ms
     Changes:   
              ----------
              diff:
                  New file
----------
          ID: maas_region_syncdb
    Function: cmd.run
        Name: maas-region syncdb --noinput
      Result: True
     Comment: Command "maas-region syncdb --noinput" run
     Started: 17:11:44.983669
    Duration: 9130.31 ms
     Changes:   
              ----------
              pid:
                  3771
              retcode:
                  0
              stderr:
              stdout:
                  Operations to perform:
                    Synchronize unmigrated apps: messages, staticfiles
                    Apply all migrations: maasserver, auth, sessions, piston3, sites, contenttypes, metadataserver
                  Synchronizing apps without migrations:
                    Creating tables...
                      Running deferred SQL...
                    Installing custom SQL...
                  Running migrations:
                    No migrations to apply.
----------
          ID: maas_region_services
    Function: service.running
        Name: maas-regiond
      Result: True
     Comment: Service restarted
     Started: 17:11:54.394271
    Duration: 232.317 ms
     Changes:   
              ----------
              maas-regiond:
                  True
----------
          ID: maas_region_services
    Function: service.running
        Name: bind9
      Result: True
     Comment: Service restarted
     Started: 17:11:54.990101
    Duration: 647.986 ms
     Changes:   
              ----------
              bind9:
                  True
----------
          ID: maas_region_services
    Function: service.running
        Name: apache2
      Result: True
     Comment: Service restarted
     Started: 17:11:55.883427
    Duration: 2126.564 ms
     Changes:   
              ----------
              apache2:
                  True
----------
          ID: maas_warmup
    Function: module.run
        Name: maasng.wait_for_http_code
      Result: True
     Comment: Module function maasng.wait_for_http_code executed
     Started: 17:11:58.012682
    Duration: 11128.305 ms
     Changes:   
              ----------
              ret:
                  ----------
                  comment:
                      MAAS API:http://localhost:5240/MAAS up.
                  result:
                      True
----------
          ID: maas_set_admin_password
    Function: cmd.run
        Name: maas createadmin --username opnfv --password opnfv_secret --email email@example.com && touch /var/lib/maas/.setup_admin
      Result: True
     Comment: Command "maas createadmin --username opnfv --password opnfv_secret --email email@example.com && touch /var/lib/maas/.setup_admin" run
     Started: 17:12:09.142474
    Duration: 5802.87 ms
     Changes:   
              ----------
              pid:
                  4253
              retcode:
                  0
              stderr:
              stdout:
----------
          ID: maas_login_admin
    Function: cmd.run
        Name: maas-region apikey --username opnfv > /var/lib/maas/.maas_credentials
      Result: True
     Comment: Command "maas-region apikey --username opnfv > /var/lib/maas/.maas_credentials" run
     Started: 17:12:14.946939
    Duration: 5587.307 ms
     Changes:   
              ----------
              pid:
                  4293
              retcode:
                  0
              stderr:
              stdout:
----------
          ID: maas_region_boot_source_resources_mirror
    Function: maasng.boot_source_present
      Result: True
     Comment: boot-source http://images.maas.io/ephemeral-v3/daily presented
     Started: 17:12:20.594658
    Duration: 101257.171 ms
     Changes:   
              ----------
              changes:
                  ----------
                  new:
                      Boot-resource http://images.maas.io/ephemeral-v3/daily/ deleted
              new:
                  boot resource http://images.maas.io/ephemeral-v3/daily was created
    Warnings: 'name' is an invalid keyword argument for
              'maasng.boot_source_present'. If you were trying to pass
              additional data to be used in a template context, please populate
              'context' with 'key: value' pairs. Your approach will work until
              Salt Fluorine is out. Please update your state files.
----------
          ID: maas_wait_for_region_import_done
    Function: module.run
        Name: maasng.boot_resources_import
      Result: True
     Comment: Module function maasng.boot_resources_import executed
     Started: 17:14:01.853677
    Duration: 26437.458 ms
     Changes:   
              ----------
              ret:
                  True
----------
          ID: maas_region_boot_sources_selection_xenial
    Function: maasng.boot_sources_selections_present
      Result: True
     Comment: boot-source selection for http://images.maas.io/ephemeral-v3/daily was created
     Started: 17:14:28.292940
    Duration: 547570.884 ms
     Changes:   
    Warnings: 'name' is an invalid keyword argument for
              'maasng.boot_sources_selections_present'. If you were trying to
              pass additional data to be used in a template context, please
              populate 'context' with 'key: value' pairs. Your approach will
              work until Salt Fluorine is out. Please update your state files.
----------
          ID: maas_wait_for_racks_import_done
    Function: module.run
        Name: maasng.sync_and_wait_bs_to_all_racks
      Result: True
     Comment: Module function maasng.sync_and_wait_bs_to_all_racks executed
     Started: 17:23:35.866946
    Duration: 85810.684 ms
     Changes:   
              ----------
              ret:
                  True
----------
          ID: maas_config
    Function: module.run
        Name: maas.process_maas_config
      Result: True
     Comment: Module function maas.process_maas_config executed
     Started: 17:25:01.680496
    Duration: 10604.698 ms
     Changes:   
              ----------
              ret:
                  ----------
                  errors:
                      ----------
                  success:
                      - enable_http_proxy
                      - upstream_dns
                      - commissioning_distro_series
                      - default_osystem
                      - active_discovery_interval
                      - dnssec_validation
                      - maas_name
                      - network_discovery
                      - enable_third_party_drivers
                      - default_storage_layout
                      - ntp_external_only
                      - disk_erase_with_secure_erase
                      - default_distro_series
                      - default_min_hwe_kernel
                  updated:
----------
          ID: maas_fabrics_pxe_admin
    Function: maasng.fabric_present
        Name: pxe_admin
      Result: True
     Comment: Module function maasng.fabric_present executed
     Started: 17:25:12.286116
    Duration: 776.158 ms
     Changes:   
              ----------
              new:
                  Fabric  pxe_admin created
              result:
                  True
----------
          ID: maas_vlan0_present_for_pxe_admin
    Function: maasng.vlan_present_in_fabric
        Name: vlan 0
      Result: True
     Comment: Vlan vlan 0 has been updated for pxe_admin
     Started: 17:25:13.062605
    Duration: 2238.812 ms
     Changes:   
              ----------
              new:
                  Vlan untagged was updated
----------
          ID: maas_create_subnet_10.1.0.0/24
    Function: maasng.subnet_present
        Name: 10.1.0.0/24
      Result: True
     Comment: Subnet 10.1.0.0/24 has been updated for pxe_admin
     Started: 17:25:15.302715
    Duration: 1292.96 ms
     Changes:   
----------
          ID: maas_create_iprange_1
    Function: maasng.iprange_present
      Result: True
     Comment: Module function maasng.iprange_present executed
     Started: 17:25:16.596975
    Duration: 293.314 ms
     Changes:   
              ----------
              new:
                  Iprange: Type:dynamic: 10.1.0.39-10.1.0.254 has been created
----------
          ID: maas_vlan0_present_for_pxe_admin_dhcp
    Function: maasng.vlan_present_in_fabric
        Name: vlan 0
      Result: True
     Comment: Vlan vlan 0 has been updated for pxe_admin
     Started: 17:25:16.890597
    Duration: 2750.604 ms
     Changes:   
              ----------
              new:
                  Vlan untagged was updated
----------
          ID: maas_sshkey_1
    Function: maasng.sshkey_present
        Name: opnfv
      Result: True
     Comment: SSH-key ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9iteXSOD1Du/fG2coTPKPKmlyjD5EHa0wwZm9pdya5BuWatdcIzkCgKLdLBST3LZZ6U8IB83LnCwKa7CX87OPxZVylK8W8NlzsjYt3P+ZOX8qUmhTxxkJa4i43aZXbQwv1a6QAwHY6n2/8mKdLpov5gf6Y28HpUN1d3mp3gegNK5quTNAxS2MzUXk9gOZ8ZXYLXCYjg4eWXXXCLxK+rmojXgsjdY3WruVLR+1YkXX38jJ0pirCp10knC+QGf1QTQE2quzYQNh8NQA+1weNUWi6xYd8c9Nt/bBdtoL56hqivJeNYiFw2PLeoGqfOtI1OrwR1Cp0WdoJ+cIdW9P/7ov has been added for user opnfv
     Started: 17:25:19.641982
    Duration: 1023.209 ms
     Changes:   
              ----------
              new:
                  SSH Key ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9iteXSOD1Du/fG2coTPKPKmlyjD5EHa0wwZm9pdya5BuWatdcIzkCgKLdLBST3LZZ6U8IB83LnCwKa7CX87OPxZVylK8W8NlzsjYt3P+ZOX8qUmhTxxkJa4i43aZXbQwv1a6QAwHY6n2/8mKdLpov5gf6Y28HpUN1d3mp3gegNK5quTNAxS2MzUXk9gOZ8ZXYLXCYjg4eWXXXCLxK+rmojXgsjdY3WruVLR+1YkXX38jJ0pirCp10knC+QGf1QTQE2quzYQNh8NQA+1weNUWi6xYd8c9Nt/bBdtoL56hqivJeNYiFw2PLeoGqfOtI1OrwR1Cp0WdoJ+cIdW9P/7ov was added.

Summary for mas01.mcp-odl-noha.local
-------------
Succeeded: 32 (changed=28)
Failed:     0
Warnings:   2
-------------
Total states run:     32
Total run time:  817.531 s
[wait_for] OK: salt -C 'mas01*' state.apply maas.region
mas01.mcp-odl-noha.local:
----------
          ID: maas_login_admin
    Function: cmd.run
        Name: maas-region apikey --username opnfv > /var/lib/maas/.maas_credentials
      Result: True
     Comment: Command "maas-region apikey --username opnfv > /var/lib/maas/.maas_credentials" run
     Started: 17:25:36.637570
    Duration: 4959.786 ms
     Changes:   
              ----------
              pid:
                  6334
              retcode:
                  0
              stderr:
              stdout:
----------
          ID: create__machines
    Function: module.run
        Name: maas.process_machines
      Result: True
     Comment: Module function maas.process_machines executed
     Started: 17:25:41.600219
    Duration: 26975.858 ms
     Changes:   
              ----------
              ret:
                  ----------
                  errors:
                      ----------
                  success:
                      - cmp002
                      - cmp003
                      - cmp001
                      - odl01
                      - gtw01
                      - ctl01
                  updated:

Summary for mas01.mcp-odl-noha.local
------------
Succeeded: 2 (changed=2)
Failed:    0
------------
Total states run:     2
Total run time:  31.936 s
[NOTE] MaaS operations might take a long time, please be patient
tput: unknown terminal "unknown"
tput: unknown terminal "unknown"
ERROR: Minions returned with non-zero exit code
mas01.mcp-odl-noha.local:
----------
          ID: maas_login_admin
    Function: cmd.run
        Name: maas-region apikey --username opnfv > /var/lib/maas/.maas_credentials
      Result: True
     Comment: Command "maas-region apikey --username opnfv > /var/lib/maas/.maas_credentials" run
     Started: 17:26:53.816728
    Duration: 5918.358 ms
     Changes:   
              ----------
              pid:
                  6765
              retcode:
                  0
              stderr:
              stdout:
----------
          ID: wait_for_machines_ready_or_deployed
    Function: module.run
        Name: maas.wait_for_machine_status
      Result: False
     Comment: Module function maas.wait_for_machine_status threw an exception. Exception: Machines:['cmp002', 'cmp003', 'cmp001', 'odl01', 'gtw01', 'ctl01']not in Ready|Deployed state
     Started: 17:26:59.739217
    Duration: 1517928.681 ms
     Changes:   

Summary for mas01.mcp-odl-noha.local
------------
Succeeded: 1 (changed=1)
Failed:    1
------------
Total states run:     2
Total run time: 1523.847 s
[wait_for] Attempt 2/5 for: ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i /var/lib/opnfv/mcp.rsa ubuntu@10.20.0.2 sudo             CI_DEBUG=false ERASE_ENV=0             /root/fuel/mcp/config/states/maas
Warning: Permanently added '10.20.0.2' (ECDSA) to the list of known hosts.

[wait_for] Waiting for cmd to return success: salt --out yaml -C 'mas01*' service.status maas-fixup | fgrep -q 'false'
[wait_for] Attempt 1/60 for: salt --out yaml -C 'mas01*' service.status maas-fixup | fgrep -q 'false'
[wait_for] OK: salt --out yaml -C 'mas01*' service.status maas-fixup | fgrep -q 'false'

[wait_for] Waiting for cmd to return success: salt -C 'mas01*' state.apply salt,iptables
[wait_for] Attempt 1/10 for: salt -C 'mas01*' state.apply salt,iptables
mas01.mcp-odl-noha.local:
----------
          ID: salt_minion_packages
    Function: pkg.installed
        Name: salt-minion
      Result: True
     Comment: All specified packages are already installed
     Started: 17:52:58.171791
    Duration: 353.792 ms
     Changes:   
----------
          ID: salt_minion_dependency_packages
    Function: pkg.installed
      Result: True
     Comment: All specified packages are already installed
     Started: 17:52:58.526123
    Duration: 14.23 ms
     Changes:   
----------
          ID: /etc/salt/minion.d/minion.conf
    Function: file.managed
      Result: True
     Comment: File /etc/salt/minion.d/minion.conf is in the correct state
     Started: 17:52:58.546745
    Duration: 575.652 ms
     Changes:   
----------
          ID: salt_minion_systemd_override
    Function: file.managed
        Name: /etc/systemd/system/salt-minion.service.d/50-restarts.conf
      Result: True
     Comment: File /etc/systemd/system/salt-minion.service.d/50-restarts.conf is in the correct state
     Started: 17:52:59.127751
    Duration: 44.426 ms
     Changes:   
----------
          ID: salt_minion_systemd_reload
    Function: module.wait
        Name: service.systemctl_reload
      Result: True
     Comment: State was not run because none of the onchanges reqs changed
     Started: 17:52:59.172914
    Duration: 0.017 ms
     Changes:   
----------
          ID: salt_minion_service
    Function: service.running
        Name: salt-minion
      Result: True
     Comment: The service salt-minion is already running
     Started: 17:52:59.173680
    Duration: 213.223 ms
     Changes:   
----------
          ID: salt_minion_sync_all
    Function: module.run
        Name: saltutil.sync_all
      Result: True
     Comment: State was not run because none of the onchanges reqs changed
     Started: 17:52:59.388805
    Duration: 0.037 ms
     Changes:   
----------
          ID: salt_minion_grains_dir
    Function: file.directory
        Name: /etc/salt/grains.d
      Result: True
     Comment: Directory /etc/salt/grains.d is in the correct state
              Directory /etc/salt/grains.d updated
     Started: 17:52:59.389191
    Duration: 2.501 ms
     Changes:   
----------
          ID: salt_minion_grains_files
    Function: file.managed
        Name: /etc/salt/grains
      Result: True
     Comment: File /etc/salt/grains exists with proper permissions. No changes made.
     Started: 17:52:59.392845
    Duration: 1.51 ms
     Changes:   
----------
          ID: salt_minion_grains_files
    Function: file.managed
        Name: /etc/salt/grains.d/placeholder
      Result: True
     Comment: File /etc/salt/grains.d/placeholder exists with proper permissions. No changes made.
     Started: 17:52:59.394964
    Duration: 1.391 ms
     Changes:   
----------
          ID: salt_minion_grain_linux_sphinx
    Function: file.managed
        Name: /etc/salt/grains.d/sphinx
      Result: True
     Comment: File /etc/salt/grains.d/sphinx is in the correct state
     Started: 17:52:59.396996
    Duration: 2.478 ms
     Changes:   
----------
          ID: salt_minion_grain_linux_sphinx_validity_check
    Function: cmd.wait
        Name: python -c "import yaml; stream = file('/etc/salt/grains.d/sphinx', 'r'); yaml.load(stream); stream.close()"
      Result: True
     Comment: 
     Started: 17:52:59.411717
    Duration: 1.556 ms
     Changes:   
----------
          ID: salt_minion_grain_linux_dns_records
    Function: file.managed
        Name: /etc/salt/grains.d/dns_records
      Result: True
     Comment: File /etc/salt/grains.d/dns_records is in the correct state
     Started: 17:52:59.413953
    Duration: 2.421 ms
     Changes:   
----------
          ID: salt_minion_grain_linux_dns_records_validity_check
    Function: cmd.wait
        Name: python -c "import yaml; stream = file('/etc/salt/grains.d/dns_records', 'r'); yaml.load(stream); stream.close()"
      Result: True
     Comment: 
     Started: 17:52:59.417791
    Duration: 1.192 ms
     Changes:   
----------
          ID: salt_minion_grain_salt_salt
    Function: file.managed
        Name: /etc/salt/grains.d/salt
      Result: True
     Comment: File /etc/salt/grains.d/salt is in the correct state
     Started: 17:52:59.419606
    Duration: 2.454 ms
     Changes:   
----------
          ID: salt_minion_grain_salt_salt_validity_check
    Function: cmd.wait
        Name: python -c "import yaml; stream = file('/etc/salt/grains.d/salt', 'r'); yaml.load(stream); stream.close()"
      Result: True
     Comment: 
     Started: 17:52:59.423346
    Duration: 1.246 ms
     Changes:   
----------
          ID: salt_minion_grains_file
    Function: cmd.wait
        Name: cat /etc/salt/grains.d/* > /etc/salt/grains
      Result: True
     Comment: 
     Started: 17:52:59.427175
    Duration: 1.531 ms
     Changes:   
----------
          ID: salt_minion_grains_publish
    Function: module.wait
        Name: mine.update
      Result: True
     Comment: 
     Started: 17:52:59.429694
    Duration: 1.001 ms
     Changes:   
----------
          ID: salt_ca_certificates_packages
    Function: pkg.installed
        Name: ca-certificates
      Result: True
     Comment: All specified packages are already installed
     Started: 17:52:59.431007
    Duration: 18.283 ms
     Changes:   
----------
          ID: salt_update_certificates
    Function: cmd.wait
        Name: update-ca-certificates
      Result: True
     Comment: 
     Started: 17:52:59.451195
    Duration: 1.378 ms
     Changes:   
----------
          ID: iptables_packages_v4
    Function: pkg.installed
        Name: iptables
      Result: True
     Comment: All specified packages are already installed
     Started: 17:52:59.452921
    Duration: 45.041 ms
     Changes:   
----------
          ID: iptables_packages_v4
    Function: pkg.installed
        Name: iptables-persistent
      Result: True
     Comment: All specified packages are already installed
     Started: 17:52:59.498313
    Duration: 29.673 ms
     Changes:   
----------
          ID: iptables_modules_v4_load
    Function: kmod.present
      Result: True
     Comment: Kernel modules iptable_filter, ip_tables are already present
     Started: 17:52:59.529760
    Duration: 37.551 ms
     Changes:   
----------
          ID: /etc/iptables/rules.v4
    Function: file.managed
      Result: True
     Comment: File /etc/iptables/rules.v4 is in the correct state
     Started: 17:52:59.568361
    Duration: 288.312 ms
     Changes:   
----------
          ID: iptables_services_v4_start
    Function: cmd.run
        Name: find /usr/share/netfilter-persistent/plugins.d/[0-9]*-ip4tables -exec {} start \;
      Result: True
     Comment: onlyif execution failed
     Started: 17:52:59.858352
    Duration: 91.859 ms
     Changes:   
----------
          ID: netfilter-persistent
    Function: service.running
      Result: True
     Comment: The service netfilter-persistent is already running
     Started: 17:52:59.951697
    Duration: 218.843 ms
     Changes:   
----------
          ID: iptables_tables_cleanup_v4
    Function: module.wait
        Name: iptables_extra.remove_stale_tables
      Result: True
     Comment: 
     Started: 17:53:00.171873
    Duration: 1.477 ms
     Changes:   
----------
          ID: iptables_services_v6_stop
    Function: cmd.run
        Name: find /usr/share/netfilter-persistent/plugins.d/[0-9]*-ip6tables -exec {} flush \;
      Result: True
     Comment: onlyif execution failed
     Started: 17:53:00.173693
    Duration: 44.838 ms
     Changes:   
----------
          ID: /etc/iptables/rules.v6
    Function: file.absent
      Result: True
     Comment: File /etc/iptables/rules.v6 is not present
     Started: 17:53:00.220131
    Duration: 1.673 ms
     Changes:   
----------
          ID: iptables_tables_flush_v6
    Function: module.wait
        Name: iptables_extra.flush_all
      Result: True
     Comment: 
     Started: 17:53:00.222885
    Duration: 1.153 ms
     Changes:   
----------
          ID: salt_minion_service_restart
    Function: cmd.run
        Name: while true; do salt-call saltutil.running|grep fun: && continue; salt-call --local service.restart salt-minion; break; done
      Result: True
     Comment: State was not run because none of the onchanges reqs changed
     Started: 17:53:00.225451
    Duration: 0.026 ms
     Changes:   

Summary for mas01.mcp-odl-noha.local
-------------
Succeeded: 31
Failed:     0
-------------
Total states run:     31
Total run time:    2.001 s
[wait_for] OK: salt -C 'mas01*' state.apply salt,iptables
mas01.mcp-odl-noha.local:
----------
          ID: maas_cluster_packages
    Function: pkg.installed
        Name: maas-rack-controller
      Result: True
     Comment: All specified packages are already installed
     Started: 17:53:15.955531
    Duration: 242.217 ms
     Changes:   
----------
          ID: maas_cluster_packages
    Function: pkg.installed
        Name: ipmitool
      Result: True
     Comment: All specified packages are already installed
     Started: 17:53:16.199018
    Duration: 18.949 ms
     Changes:   
----------
          ID: /etc/maas/rackd.conf
    Function: file.line
      Result: True
     Comment: No changes needed to be made
     Started: 17:53:16.229638
    Duration: 5.52 ms
     Changes:   
----------
          ID: /etc/maas/rackd_conf
    Function: file.managed
        Name: /etc/maas/rackd.conf
      Result: True
     Comment: File /etc/maas/rackd.conf exists with proper permissions. No changes made.
     Started: 17:53:16.236214
    Duration: 21.834 ms
     Changes:   
----------
          ID: maas_cluster_services
    Function: service.running
        Name: maas-rackd
      Result: True
     Comment: The service maas-rackd is already running
     Started: 17:53:16.261162
    Duration: 234.103 ms
     Changes:   

Summary for mas01.mcp-odl-noha.local
------------
Succeeded: 5
Failed:    0
------------
Total states run:     5
Total run time: 522.623 ms

[wait_for] Waiting for cmd to return success: salt -C 'mas01*' state.apply maas.region
[wait_for] Attempt 1/10 for: salt -C 'mas01*' state.apply maas.region
mas01.mcp-odl-noha.local:
----------
          ID: maas_region_packages
    Function: pkg.installed
        Name: maas-region-controller
      Result: True
     Comment: All specified packages are already installed
     Started: 17:53:22.968574
    Duration: 431.788 ms
     Changes:   
----------
          ID: maas_region_packages
    Function: pkg.installed
        Name: python-oauth
      Result: True
     Comment: All specified packages are already installed
     Started: 17:53:23.400880
    Duration: 34.38 ms
     Changes:   
----------
          ID: /etc/maas/regiond.conf
    Function: file.replace
      Result: True
     Comment: No changes needed to be made
     Started: 17:53:23.441277
    Duration: 32.095 ms
     Changes:   
----------
          ID: /usr/lib/python3/dist-packages/provisioningserver/templates/proxy/maas-proxy.conf.template
    Function: file.managed
      Result: True
     Comment: File /usr/lib/python3/dist-packages/provisioningserver/templates/proxy/maas-proxy.conf.template is in the correct state
     Started: 17:53:23.474074
    Duration: 498.15 ms
     Changes:   
----------
          ID: maas_timeout_commissioning
    Function: file.replace
        Name: /usr/lib/python3/dist-packages/maasserver/node_status.py
      Result: True
     Comment: No changes needed to be made
     Started: 17:53:23.972947
    Duration: 20.591 ms
     Changes:   
----------
          ID: maas_timeout_deploying
    Function: file.replace
        Name: /usr/lib/python3/dist-packages/maasserver/node_status.py
      Result: True
     Comment: No changes needed to be made
     Started: 17:53:23.994436
    Duration: 6.342 ms
     Changes:   
----------
          ID: maas_interface_default_mode_dhcp
    Function: file.replace
        Name: /usr/lib/python3/dist-packages/maasserver/models/node.py
      Result: True
     Comment: No changes needed to be made
     Started: 17:53:24.001513
    Duration: 136.59 ms
     Changes:   
----------
          ID: /etc/apache2/conf-enabled/maas-http.conf
    Function: file.managed
      Result: True
     Comment: File /etc/apache2/conf-enabled/maas-http.conf is in the correct state
     Started: 17:53:24.138878
    Duration: 176.424 ms
     Changes:   
----------
          ID: maas_apache_headers
    Function: cmd.run
        Name: a2enmod headers
      Result: True
     Comment: Command "a2enmod headers" run
     Started: 17:53:24.318906
    Duration: 276.123 ms
     Changes:   
              ----------
              pid:
                  12363
              retcode:
                  0
              stderr:
              stdout:
                  Module headers already enabled
----------
          ID: /usr/share/maas/web/static/css/maas-styles.css
    Function: file.managed
      Result: True
     Comment: File /usr/share/maas/web/static/css/maas-styles.css is in the correct state
     Started: 17:53:24.595669
    Duration: 190.554 ms
     Changes:   
----------
          ID: /etc/maas/preseeds/curtin_userdata_amd64_generic_trusty
    Function: file.managed
      Result: True
     Comment: File /etc/maas/preseeds/curtin_userdata_amd64_generic_trusty is in the correct state
     Started: 17:53:24.787148
    Duration: 348.677 ms
     Changes:   
----------
          ID: /etc/maas/preseeds/curtin_userdata_amd64_generic_xenial
    Function: file.managed
      Result: True
     Comment: File /etc/maas/preseeds/curtin_userdata_amd64_generic_xenial is in the correct state
     Started: 17:53:25.137273
    Duration: 304.271 ms
     Changes:   
----------
          ID: /etc/maas/preseeds/curtin_userdata_arm64_generic_xenial
    Function: file.managed
      Result: True
     Comment: File /etc/maas/preseeds/curtin_userdata_arm64_generic_xenial is in the correct state
     Started: 17:53:25.443218
    Duration: 353.672 ms
     Changes:   
----------
          ID: Configure /root/.pgpass for MAAS
    Function: file.managed
        Name: /root/.pgpass
      Result: True
     Comment: File /root/.pgpass is in the correct state
     Started: 17:53:25.798044
    Duration: 362.17 ms
     Changes:   
----------
          ID: maas_region_syncdb
    Function: cmd.run
        Name: maas-region syncdb --noinput
      Result: True
     Comment: Command "maas-region syncdb --noinput" run
     Started: 17:53:26.168311
    Duration: 7827.482 ms
     Changes:   
              ----------
              pid:
                  12376
              retcode:
                  0
              stderr:
              stdout:
                  Operations to perform:
                    Synchronize unmigrated apps: messages, staticfiles
                    Apply all migrations: maasserver, sites, sessions, piston3, contenttypes, auth, metadataserver
                  Synchronizing apps without migrations:
                    Creating tables...
                      Running deferred SQL...
                    Installing custom SQL...
                  Running migrations:
                    No migrations to apply.
----------
          ID: maas_region_services
    Function: service.running
        Name: maas-regiond
      Result: True
     Comment: The service maas-regiond is already running
     Started: 17:53:33.998650
    Duration: 207.834 ms
     Changes:   
----------
          ID: maas_region_services
    Function: service.running
        Name: bind9
      Result: True
     Comment: The service bind9 is already running
     Started: 17:53:34.208957
    Duration: 98.487 ms
     Changes:   
----------
          ID: maas_region_services
    Function: service.running
        Name: apache2
      Result: True
     Comment: The service apache2 is already running
     Started: 17:53:34.309875
    Duration: 146.585 ms
     Changes:   
----------
          ID: maas_warmup
    Function: module.run
        Name: maasng.wait_for_http_code
      Result: True
     Comment: Module function maasng.wait_for_http_code executed
     Started: 17:53:34.458723
    Duration: 22.118 ms
     Changes:   
              ----------
              ret:
                  ----------
                  comment:
                      MAAS API:http://localhost:5240/MAAS up.
                  result:
                      True
----------
          ID: maas_set_admin_password
    Function: cmd.run
        Name: maas createadmin --username opnfv --password opnfv_secret --email email@example.com && touch /var/lib/maas/.setup_admin
      Result: True
     Comment: /var/lib/maas/.setup_admin exists
     Started: 17:53:34.482026
    Duration: 1.395 ms
     Changes:   
----------
          ID: maas_login_admin
    Function: cmd.run
        Name: maas-region apikey --username opnfv > /var/lib/maas/.maas_credentials
      Result: True
     Comment: Command "maas-region apikey --username opnfv > /var/lib/maas/.maas_credentials" run
     Started: 17:53:34.484484
    Duration: 5455.444 ms
     Changes:   
              ----------
              pid:
                  12418
              retcode:
                  0
              stderr:
              stdout:
----------
          ID: maas_region_boot_source_resources_mirror
    Function: maasng.boot_source_present
      Result: True
     Comment: boot-source http://images.maas.io/ephemeral-v3/daily alredy exist
     Started: 17:53:39.948312
    Duration: 367.266 ms
     Changes:   
              ----------
              changes:
                  ----------
    Warnings: 'name' is an invalid keyword argument for
              'maasng.boot_source_present'. If you were trying to pass
              additional data to be used in a template context, please populate
              'context' with 'key: value' pairs. Your approach will work until
              Salt Fluorine is out. Please update your state files.
----------
          ID: maas_wait_for_region_import_done
    Function: module.run
        Name: maasng.boot_resources_import
      Result: True
     Comment: Module function maasng.boot_resources_import executed
     Started: 17:53:40.316625
    Duration: 31512.467 ms
     Changes:   
              ----------
              ret:
                  True
----------
          ID: maas_region_boot_sources_selection_xenial
    Function: maasng.boot_sources_selections_present
      Result: True
     Comment: Requested boot-source selection for http://images.maas.io/ephemeral-v3/daily already exist.
     Started: 17:54:11.830926
    Duration: 823.731 ms
     Changes:   
    Warnings: 'name' is an invalid keyword argument for
              'maasng.boot_sources_selections_present'. If you were trying to
              pass additional data to be used in a template context, please
              populate 'context' with 'key: value' pairs. Your approach will
              work until Salt Fluorine is out. Please update your state files.
----------
          ID: maas_wait_for_racks_import_done
    Function: module.run
        Name: maasng.sync_and_wait_bs_to_all_racks
      Result: True
     Comment: Module function maasng.sync_and_wait_bs_to_all_racks executed
     Started: 17:54:12.655851
    Duration: 4040.947 ms
     Changes:   
              ----------
              ret:
                  True
----------
          ID: maas_config
    Function: module.run
        Name: maas.process_maas_config
      Result: True
     Comment: Module function maas.process_maas_config executed
     Started: 17:54:16.698749
    Duration: 8673.313 ms
     Changes:   
              ----------
              ret:
                  ----------
                  errors:
                      ----------
                  success:
                      - enable_http_proxy
                      - upstream_dns
                      - commissioning_distro_series
                      - default_osystem
                      - active_discovery_interval
                      - dnssec_validation
                      - maas_name
                      - network_discovery
                      - enable_third_party_drivers
                      - default_storage_layout
                      - ntp_external_only
                      - disk_erase_with_secure_erase
                      - default_distro_series
                      - default_min_hwe_kernel
                  updated:
----------
          ID: maas_fabrics_pxe_admin
    Function: maasng.fabric_present
        Name: pxe_admin
      Result: True
     Comment: Module function maasng.fabric_present executed
     Started: 17:54:25.394124
    Duration: 1340.349 ms
     Changes:   
              ----------
              new:
                  Fabric  pxe_admin created
              result:
                  True
----------
          ID: maas_vlan0_present_for_pxe_admin
    Function: maasng.vlan_present_in_fabric
        Name: vlan 0
      Result: True
     Comment: Vlan vlan 0 has been updated for pxe_admin
     Started: 17:54:26.735082
    Duration: 2848.661 ms
     Changes:   
              ----------
              new:
                  Vlan untagged was updated
----------
          ID: maas_create_subnet_10.1.0.0/24
    Function: maasng.subnet_present
        Name: 10.1.0.0/24
      Result: True
     Comment: Subnet 10.1.0.0/24 has been updated for pxe_admin
     Started: 17:54:29.589682
    Duration: 2385.977 ms
     Changes:   
----------
          ID: maas_create_iprange_1
    Function: maasng.iprange_present
      Result: True
     Comment: Iprange maas_create_iprange_1 already exist.
     Started: 17:54:31.981614
    Duration: 433.105 ms
     Changes:   
----------
          ID: maas_vlan0_present_for_pxe_admin_dhcp
    Function: maasng.vlan_present_in_fabric
        Name: vlan 0
      Result: True
     Comment: Vlan vlan 0 has been updated for pxe_admin
     Started: 17:54:32.415178
    Duration: 3736.438 ms
     Changes:   
              ----------
              new:
                  Vlan untagged was updated
----------
          ID: maas_sshkey_1
    Function: maasng.sshkey_present
        Name: opnfv
      Result: True
     Comment: SSH key ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9iteXSOD1Du/fG2coTPKPKmlyjD5EHa0wwZm9pdya5BuWatdcIzkCgKLdLBST3LZZ6U8IB83LnCwKa7CX87OPxZVylK8W8NlzsjYt3P+ZOX8qUmhTxxkJa4i43aZXbQwv1a6QAwHY6n2/8mKdLpov5gf6Y28HpUN1d3mp3gegNK5quTNAxS2MzUXk9gOZ8ZXYLXCYjg4eWXXXCLxK+rmojXgsjdY3WruVLR+1YkXX38jJ0pirCp10knC+QGf1QTQE2quzYQNh8NQA+1weNUWi6xYd8c9Nt/bBdtoL56hqivJeNYiFw2PLeoGqfOtI1OrwR1Cp0WdoJ+cIdW9P/7ov already exist for user opnfv.
     Started: 17:54:36.152871
    Duration: 324.455 ms
     Changes:   

Summary for mas01.mcp-odl-noha.local
-------------
Succeeded: 32 (changed=11)
Failed:     0
Warnings:   2
-------------
Total states run:     32
Total run time:   73.418 s
[wait_for] OK: salt -C 'mas01*' state.apply maas.region
mas01.mcp-odl-noha.local:
----------
          ID: maas_login_admin
    Function: cmd.run
        Name: maas-region apikey --username opnfv > /var/lib/maas/.maas_credentials
      Result: True
     Comment: Command "maas-region apikey --username opnfv > /var/lib/maas/.maas_credentials" run
     Started: 17:54:49.996832
    Duration: 5101.809 ms
     Changes:   
              ----------
              pid:
                  12890
              retcode:
                  0
              stderr:
              stdout:
----------
          ID: create__machines
    Function: module.run
        Name: maas.process_machines
      Result: True
     Comment: Module function maas.process_machines executed
     Started: 17:54:55.101631
    Duration: 17328.302 ms
     Changes:   
              ----------
              ret:
                  ----------
                  errors:
                      ----------
                  success:
                  updated:
                      - cmp002
                      - cmp003
                      - cmp001
                      - odl01
                      - gtw01
                      - ctl01

Summary for mas01.mcp-odl-noha.local
------------
Succeeded: 2 (changed=2)
Failed:    0
------------
Total states run:     2
Total run time:  22.430 s
[NOTE] MaaS operations might take a long time, please be patient
tput: unknown terminal "unknown"
tput: unknown terminal "unknown"
mas01.mcp-odl-noha.local:
----------
          ID: maas_login_admin
    Function: cmd.run
        Name: maas-region apikey --username opnfv > /var/lib/maas/.maas_credentials
      Result: True
     Comment: Command "maas-region apikey --username opnfv > /var/lib/maas/.maas_credentials" run
     Started: 17:55:58.769712
    Duration: 5667.318 ms
     Changes:   
              ----------
              pid:
                  13216
              retcode:
                  0
              stderr:
              stdout:
----------
          ID: wait_for_machines_ready_or_deployed
    Function: module.run
        Name: maas.wait_for_machine_status
      Result: False
     Comment: Module function maas.wait_for_machine_status threw an exception. Exception: Machines:['cmp002', 'cmp003', 'cmp001', 'odl01', 'gtw01', 'ctl01']not in Ready|Deployed state
     Started: 17:56:04.444118
    Duration: 1505044.442 ms
     Changes:   

Summary for mas01.mcp-odl-noha.local
------------
Succeeded: 1 (changed=1)
Failed:    1
------------
Total states run:     2
Total run time: 1510.712 s
ERROR: Minions returned with non-zero exit code
[wait_for] Attempt 3/5 for: ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i /var/lib/opnfv/mcp.rsa ubuntu@10.20.0.2 sudo             CI_DEBUG=false ERASE_ENV=0             /root/fuel/mcp/config/states/maas
Warning: Permanently added '10.20.0.2' (ECDSA) to the list of known hosts.

[wait_for] Waiting for cmd to return success: salt --out yaml -C 'mas01*' service.status maas-fixup | fgrep -q 'false'
[wait_for] Attempt 1/60 for: salt --out yaml -C 'mas01*' service.status maas-fixup | fgrep -q 'false'
[wait_for] OK: salt --out yaml -C 'mas01*' service.status maas-fixup | fgrep -q 'false'

[wait_for] Waiting for cmd to return success: salt -C 'mas01*' state.apply salt,iptables
[wait_for] Attempt 1/10 for: salt -C 'mas01*' state.apply salt,iptables
mas01.mcp-odl-noha.local:
----------
          ID: salt_minion_packages
    Function: pkg.installed
        Name: salt-minion
      Result: True
     Comment: All specified packages are already installed
     Started: 18:21:40.903000
    Duration: 178.263 ms
     Changes:   
----------
          ID: salt_minion_dependency_packages
    Function: pkg.installed
      Result: True
     Comment: All specified packages are already installed
     Started: 18:21:41.081744
    Duration: 13.705 ms
     Changes:   
----------
          ID: /etc/salt/minion.d/minion.conf
    Function: file.managed
      Result: True
     Comment: File /etc/salt/minion.d/minion.conf is in the correct state
     Started: 18:21:41.101590
    Duration: 443.944 ms
     Changes:   
----------
          ID: salt_minion_systemd_override
    Function: file.managed
        Name: /etc/systemd/system/salt-minion.service.d/50-restarts.conf
      Result: True
     Comment: File /etc/systemd/system/salt-minion.service.d/50-restarts.conf is in the correct state
     Started: 18:21:41.562754
    Duration: 44.17 ms
     Changes:   
----------
          ID: salt_minion_systemd_reload
    Function: module.wait
        Name: service.systemctl_reload
      Result: True
     Comment: State was not run because none of the onchanges reqs changed
     Started: 18:21:41.607410
    Duration: 0.011 ms
     Changes:   
----------
          ID: salt_minion_service
    Function: service.running
        Name: salt-minion
      Result: True
     Comment: The service salt-minion is already running
     Started: 18:21:41.607883
    Duration: 133.084 ms
     Changes:   
----------
          ID: salt_minion_sync_all
    Function: module.run
        Name: saltutil.sync_all
      Result: True
     Comment: State was not run because none of the onchanges reqs changed
     Started: 18:21:41.742865
    Duration: 0.034 ms
     Changes:   
----------
          ID: salt_minion_grains_dir
    Function: file.directory
        Name: /etc/salt/grains.d
      Result: True
     Comment: Directory /etc/salt/grains.d is in the correct state
              Directory /etc/salt/grains.d updated
     Started: 18:21:41.743242
    Duration: 2.543 ms
     Changes:   
----------
          ID: salt_minion_grains_files
    Function: file.managed
        Name: /etc/salt/grains
      Result: True
     Comment: File /etc/salt/grains exists with proper permissions. No changes made.
     Started: 18:21:41.746809
    Duration: 1.545 ms
     Changes:   
----------
          ID: salt_minion_grains_files
    Function: file.managed
        Name: /etc/salt/grains.d/placeholder
      Result: True
     Comment: File /etc/salt/grains.d/placeholder exists with proper permissions. No changes made.
     Started: 18:21:41.749049
    Duration: 13.724 ms
     Changes:   
----------
          ID: salt_minion_grain_linux_sphinx
    Function: file.managed
        Name: /etc/salt/grains.d/sphinx
      Result: True
     Comment: File /etc/salt/grains.d/sphinx is in the correct state
     Started: 18:21:41.763428
    Duration: 5.809 ms
     Changes:   
----------
          ID: salt_minion_grain_linux_sphinx_validity_check
    Function: cmd.wait
        Name: python -c "import yaml; stream = file('/etc/salt/grains.d/sphinx', 'r'); yaml.load(stream); stream.close()"
      Result: True
     Comment: 
     Started: 18:21:41.773941
    Duration: 1.298 ms
     Changes:   
----------
          ID: salt_minion_grain_linux_dns_records
    Function: file.managed
        Name: /etc/salt/grains.d/dns_records
      Result: True
     Comment: File /etc/salt/grains.d/dns_records is in the correct state
     Started: 18:21:41.775843
    Duration: 2.318 ms
     Changes:   
----------
          ID: salt_minion_grain_linux_dns_records_validity_check
    Function: cmd.wait
        Name: python -c "import yaml; stream = file('/etc/salt/grains.d/dns_records', 'r'); yaml.load(stream); stream.close()"
      Result: True
     Comment: 
     Started: 18:21:41.779492
    Duration: 1.234 ms
     Changes:   
----------
          ID: salt_minion_grain_salt_salt
    Function: file.managed
        Name: /etc/salt/grains.d/salt
      Result: True
     Comment: File /etc/salt/grains.d/salt is in the correct state
     Started: 18:21:41.781373
    Duration: 2.237 ms
     Changes:   
----------
          ID: salt_minion_grain_salt_salt_validity_check
    Function: cmd.wait
        Name: python -c "import yaml; stream = file('/etc/salt/grains.d/salt', 'r'); yaml.load(stream); stream.close()"
      Result: True
     Comment: 
     Started: 18:21:41.789260
    Duration: 2.566 ms
     Changes:   
----------
          ID: salt_minion_grains_file
    Function: cmd.wait
        Name: cat /etc/salt/grains.d/* > /etc/salt/grains
      Result: True
     Comment: 
     Started: 18:21:41.794528
    Duration: 1.471 ms
     Changes:   
----------
          ID: salt_minion_grains_publish
    Function: module.wait
        Name: mine.update
      Result: True
     Comment: 
     Started: 18:21:41.796998
    Duration: 1.001 ms
     Changes:   
----------
          ID: salt_ca_certificates_packages
    Function: pkg.installed
        Name: ca-certificates
      Result: True
     Comment: All specified packages are already installed
     Started: 18:21:41.798301
    Duration: 14.38 ms
     Changes:   
----------
          ID: salt_update_certificates
    Function: cmd.wait
        Name: update-ca-certificates
      Result: True
     Comment: 
     Started: 18:21:41.814581
    Duration: 1.272 ms
     Changes:   
----------
          ID: iptables_packages_v4
    Function: pkg.installed
        Name: iptables
      Result: True
     Comment: All specified packages are already installed
     Started: 18:21:41.816169
    Duration: 17.549 ms
     Changes:   
----------
          ID: iptables_packages_v4
    Function: pkg.installed
        Name: iptables-persistent
      Result: True
     Comment: All specified packages are already installed
     Started: 18:21:41.834106
    Duration: 17.417 ms
     Changes:   
----------
          ID: iptables_modules_v4_load
    Function: kmod.present
      Result: True
     Comment: Kernel modules iptable_filter, ip_tables are already present
     Started: 18:21:41.853321
    Duration: 37.991 ms
     Changes:   
----------
          ID: /etc/iptables/rules.v4
    Function: file.managed
      Result: True
     Comment: File /etc/iptables/rules.v4 is in the correct state
     Started: 18:21:41.892493
    Duration: 202.586 ms
     Changes:   
----------
          ID: iptables_services_v4_start
    Function: cmd.run
        Name: find /usr/share/netfilter-persistent/plugins.d/[0-9]*-ip4tables -exec {} start \;
      Result: True
     Comment: onlyif execution failed
     Started: 18:21:42.096735
    Duration: 42.037 ms
     Changes:   
----------
          ID: netfilter-persistent
    Function: service.running
      Result: True
     Comment: The service netfilter-persistent is already running
     Started: 18:21:42.139833
    Duration: 102.151 ms
     Changes:   
----------
          ID: iptables_tables_cleanup_v4
    Function: module.wait
        Name: iptables_extra.remove_stale_tables
      Result: True
     Comment: 
     Started: 18:21:42.243291
    Duration: 1.346 ms
     Changes:   
----------
          ID: iptables_services_v6_stop
    Function: cmd.run
        Name: find /usr/share/netfilter-persistent/plugins.d/[0-9]*-ip6tables -exec {} flush \;
      Result: True
     Comment: onlyif execution failed
     Started: 18:21:42.244976
    Duration: 37.574 ms
     Changes:   
----------
          ID: /etc/iptables/rules.v6
    Function: file.absent
      Result: True
     Comment: File /etc/iptables/rules.v6 is not present
     Started: 18:21:42.284070
    Duration: 1.61 ms
     Changes:   
----------
          ID: iptables_tables_flush_v6
    Function: module.wait
        Name: iptables_extra.flush_all
      Result: True
     Comment: 
     Started: 18:21:42.286752
    Duration: 1.016 ms
     Changes:   
----------
          ID: salt_minion_service_restart
    Function: cmd.run
        Name: while true; do salt-call saltutil.running|grep fun: && continue; salt-call --local service.restart salt-minion; break; done
      Result: True
     Comment: State was not run because none of the onchanges reqs changed
     Started: 18:21:42.289062
    Duration: 0.021 ms
     Changes:   

Summary for mas01.mcp-odl-noha.local
-------------
Succeeded: 31
Failed:     0
-------------
Total states run:     31
Total run time:    1.326 s
[wait_for] OK: salt -C 'mas01*' state.apply salt,iptables
mas01.mcp-odl-noha.local:
----------
          ID: maas_cluster_packages
    Function: pkg.installed
        Name: maas-rack-controller
      Result: True
     Comment: All specified packages are already installed
     Started: 18:21:46.850965
    Duration: 163.273 ms
     Changes:   
----------
          ID: maas_cluster_packages
    Function: pkg.installed
        Name: ipmitool
      Result: True
     Comment: All specified packages are already installed
     Started: 18:21:47.014718
    Duration: 13.736 ms
     Changes:   
----------
          ID: /etc/maas/rackd.conf
    Function: file.line
      Result: True
     Comment: No changes needed to be made
     Started: 18:21:47.038634
    Duration: 2.6 ms
     Changes:   
----------
          ID: /etc/maas/rackd_conf
    Function: file.managed
        Name: /etc/maas/rackd.conf
      Result: True
     Comment: File /etc/maas/rackd.conf exists with proper permissions. No changes made.
     Started: 18:21:47.041582
    Duration: 2.545 ms
     Changes:   
----------
          ID: maas_cluster_services
    Function: service.running
        Name: maas-rackd
      Result: True
     Comment: The service maas-rackd is already running
     Started: 18:21:47.045941
    Duration: 124.355 ms
     Changes:   

Summary for mas01.mcp-odl-noha.local
------------
Succeeded: 5
Failed:    0
------------
Total states run:     5
Total run time: 306.509 ms

[wait_for] Waiting for cmd to return success: salt -C 'mas01*' state.apply maas.region
[wait_for] Attempt 1/10 for: salt -C 'mas01*' state.apply maas.region
mas01.mcp-odl-noha.local:
----------
          ID: maas_region_packages
    Function: pkg.installed
        Name: maas-region-controller
      Result: True
     Comment: All specified packages are already installed
     Started: 18:21:58.786676
    Duration: 188.323 ms
     Changes:   
----------
          ID: maas_region_packages
    Function: pkg.installed
        Name: python-oauth
      Result: True
     Comment: All specified packages are already installed
     Started: 18:21:58.977119
    Duration: 20.257 ms
     Changes:   
----------
          ID: /etc/maas/regiond.conf
    Function: file.replace
      Result: True
     Comment: No changes needed to be made
     Started: 18:21:59.005713
    Duration: 11.91 ms
     Changes:   
----------
          ID: /usr/lib/python3/dist-packages/provisioningserver/templates/proxy/maas-proxy.conf.template
    Function: file.managed
      Result: True
     Comment: File /usr/lib/python3/dist-packages/provisioningserver/templates/proxy/maas-proxy.conf.template is in the correct state
     Started: 18:21:59.018918
    Duration: 121.157 ms
     Changes:   
----------
          ID: maas_timeout_commissioning
    Function: file.replace
        Name: /usr/lib/python3/dist-packages/maasserver/node_status.py
      Result: True
     Comment: No changes needed to be made
     Started: 18:21:59.141441
    Duration: 8.244 ms
     Changes:   
----------
          ID: maas_timeout_deploying
    Function: file.replace
        Name: /usr/lib/python3/dist-packages/maasserver/node_status.py
      Result: True
     Comment: No changes needed to be made
     Started: 18:21:59.151143
    Duration: 23.881 ms
     Changes:   
----------
          ID: maas_interface_default_mode_dhcp
    Function: file.replace
        Name: /usr/lib/python3/dist-packages/maasserver/models/node.py
      Result: True
     Comment: No changes needed to be made
     Started: 18:21:59.175775
    Duration: 40.496 ms
     Changes:   
----------
          ID: /etc/apache2/conf-enabled/maas-http.conf
    Function: file.managed
      Result: True
     Comment: File /etc/apache2/conf-enabled/maas-http.conf is in the correct state
     Started: 18:21:59.216820
    Duration: 29.541 ms
     Changes:   
----------
          ID: maas_apache_headers
    Function: cmd.run
        Name: a2enmod headers
      Result: True
     Comment: Command "a2enmod headers" run
     Started: 18:21:59.249393
    Duration: 124.184 ms
     Changes:   
              ----------
              pid:
                  19243
              retcode:
                  0
              stderr:
              stdout:
                  Module headers already enabled
----------
          ID: /usr/share/maas/web/static/css/maas-styles.css
    Function: file.managed
      Result: True
     Comment: File /usr/share/maas/web/static/css/maas-styles.css is in the correct state
     Started: 18:21:59.374139
    Duration: 40.766 ms
     Changes:   
----------
          ID: /etc/maas/preseeds/curtin_userdata_amd64_generic_trusty
    Function: file.managed
      Result: True
     Comment: File /etc/maas/preseeds/curtin_userdata_amd64_generic_trusty is in the correct state
     Started: 18:21:59.415519
    Duration: 80.618 ms
     Changes:   
----------
          ID: /etc/maas/preseeds/curtin_userdata_amd64_generic_xenial
    Function: file.managed
      Result: True
     Comment: File /etc/maas/preseeds/curtin_userdata_amd64_generic_xenial is in the correct state
     Started: 18:21:59.496700
    Duration: 110.466 ms
     Changes:   
----------
          ID: /etc/maas/preseeds/curtin_userdata_arm64_generic_xenial
    Function: file.managed
      Result: True
     Comment: File /etc/maas/preseeds/curtin_userdata_arm64_generic_xenial is in the correct state
     Started: 18:21:59.607954
    Duration: 136.516 ms
     Changes:   
----------
          ID: Configure /root/.pgpass for MAAS
    Function: file.managed
        Name: /root/.pgpass
      Result: True
     Comment: File /root/.pgpass is in the correct state
     Started: 18:21:59.744896
    Duration: 105.355 ms
     Changes:   
----------
          ID: maas_region_syncdb
    Function: cmd.run
        Name: maas-region syncdb --noinput
      Result: True
     Comment: Command "maas-region syncdb --noinput" run
     Started: 18:21:59.857583
    Duration: 6069.784 ms
     Changes:   
              ----------
              pid:
                  19259
              retcode:
                  0
              stderr:
              stdout:
                  Operations to perform:
                    Synchronize unmigrated apps: messages, staticfiles
                    Apply all migrations: sites, contenttypes, sessions, piston3, metadataserver, maasserver, auth
                  Synchronizing apps without migrations:
                    Creating tables...
                      Running deferred SQL...
                    Installing custom SQL...
                  Running migrations:
                    No migrations to apply.
----------
          ID: maas_region_services
    Function: service.running
        Name: maas-regiond
      Result: True
     Comment: The service maas-regiond is already running
     Started: 18:22:05.931051
    Duration: 182.885 ms
     Changes:   
----------
          ID: maas_region_services
    Function: service.running
        Name: bind9
      Result: True
     Comment: The service bind9 is already running
     Started: 18:22:06.115835
    Duration: 107.724 ms
     Changes:   
----------
          ID: maas_region_services
    Function: service.running
        Name: apache2
      Result: True
     Comment: The service apache2 is already running
     Started: 18:22:06.225443
    Duration: 97.411 ms
     Changes:   
----------
          ID: maas_warmup
    Function: module.run
        Name: maasng.wait_for_http_code
      Result: True
     Comment: Module function maasng.wait_for_http_code executed
     Started: 18:22:06.325677
    Duration: 23.46 ms
     Changes:   
              ----------
              ret:
                  ----------
                  comment:
                      MAAS API:http://localhost:5240/MAAS up.
                  result:
                      True
----------
          ID: maas_set_admin_password
    Function: cmd.run
        Name: maas createadmin --username opnfv --password opnfv_secret --email email@example.com && touch /var/lib/maas/.setup_admin
      Result: True
     Comment: /var/lib/maas/.setup_admin exists
     Started: 18:22:06.350294
    Duration: 1.239 ms
     Changes:   
----------
          ID: maas_login_admin
    Function: cmd.run
        Name: maas-region apikey --username opnfv > /var/lib/maas/.maas_credentials
      Result: True
     Comment: Command "maas-region apikey --username opnfv > /var/lib/maas/.maas_credentials" run
     Started: 18:22:06.352473
    Duration: 3793.238 ms
     Changes:   
              ----------
              pid:
                  19300
              retcode:
                  0
              stderr:
              stdout:
----------
          ID: maas_region_boot_source_resources_mirror
    Function: maasng.boot_source_present
      Result: True
     Comment: boot-source http://images.maas.io/ephemeral-v3/daily alredy exist
     Started: 18:22:10.158703
    Duration: 449.529 ms
     Changes:   
              ----------
              changes:
                  ----------
    Warnings: 'name' is an invalid keyword argument for
              'maasng.boot_source_present'. If you were trying to pass
              additional data to be used in a template context, please populate
              'context' with 'key: value' pairs. Your approach will work until
              Salt Fluorine is out. Please update your state files.
----------
          ID: maas_wait_for_region_import_done
    Function: module.run
        Name: maasng.boot_resources_import
      Result: True
     Comment: Module function maasng.boot_resources_import executed
     Started: 18:22:10.609942
    Duration: 25986.545 ms
     Changes:   
              ----------
              ret:
                  True
----------
          ID: maas_region_boot_sources_selection_xenial
    Function: maasng.boot_sources_selections_present
      Result: True
     Comment: Requested boot-source selection for http://images.maas.io/ephemeral-v3/daily already exist.
     Started: 18:22:36.598617
    Duration: 544.06 ms
     Changes:   
    Warnings: 'name' is an invalid keyword argument for
              'maasng.boot_sources_selections_present'. If you were trying to
              pass additional data to be used in a template context, please
              populate 'context' with 'key: value' pairs. Your approach will
              work until Salt Fluorine is out. Please update your state files.
----------
          ID: maas_wait_for_racks_import_done
    Function: module.run
        Name: maasng.sync_and_wait_bs_to_all_racks
      Result: True
     Comment: Module function maasng.sync_and_wait_bs_to_all_racks executed
     Started: 18:22:37.144403
    Duration: 2947.571 ms
     Changes:   
              ----------
              ret:
                  True
----------
          ID: maas_config
    Function: module.run
        Name: maas.process_maas_config
      Result: True
     Comment: Module function maas.process_maas_config executed
     Started: 18:22:40.094796
    Duration: 5963.532 ms
     Changes:   
              ----------
              ret:
                  ----------
                  errors:
                      ----------
                  success:
                      - enable_http_proxy
                      - upstream_dns
                      - commissioning_distro_series
                      - default_osystem
                      - active_discovery_interval
                      - dnssec_validation
                      - maas_name
                      - network_discovery
                      - enable_third_party_drivers
                      - default_storage_layout
                      - ntp_external_only
                      - disk_erase_with_secure_erase
                      - default_distro_series
                      - default_min_hwe_kernel
                  updated:
----------
          ID: maas_fabrics_pxe_admin
    Function: maasng.fabric_present
        Name: pxe_admin
      Result: True
     Comment: Module function maasng.fabric_present executed
     Started: 18:22:46.059187
    Duration: 272.142 ms
     Changes:   
              ----------
              new:
                  Fabric  pxe_admin created
              result:
                  True
----------
          ID: maas_vlan0_present_for_pxe_admin
    Function: maasng.vlan_present_in_fabric
        Name: vlan 0
      Result: True
     Comment: Vlan vlan 0 has been updated for pxe_admin
     Started: 18:22:46.331625
    Duration: 1410.971 ms
     Changes:   
              ----------
              new:
                  Vlan untagged was updated
----------
          ID: maas_create_subnet_10.1.0.0/24
    Function: maasng.subnet_present
        Name: 10.1.0.0/24
      Result: True
     Comment: Subnet 10.1.0.0/24 has been updated for pxe_admin
     Started: 18:22:47.743893
    Duration: 829.368 ms
     Changes:   
----------
          ID: maas_create_iprange_1
    Function: maasng.iprange_present
      Result: True
     Comment: Iprange maas_create_iprange_1 already exist.
     Started: 18:22:48.575217
    Duration: 232.675 ms
     Changes:   
----------
          ID: maas_vlan0_present_for_pxe_admin_dhcp
    Function: maasng.vlan_present_in_fabric
        Name: vlan 0
      Result: True
     Comment: Vlan vlan 0 has been updated for pxe_admin
     Started: 18:22:48.808296
    Duration: 1527.232 ms
     Changes:   
              ----------
              new:
                  Vlan untagged was updated
----------
          ID: maas_sshkey_1
    Function: maasng.sshkey_present
        Name: opnfv
      Result: True
     Comment: SSH key ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9iteXSOD1Du/fG2coTPKPKmlyjD5EHa0wwZm9pdya5BuWatdcIzkCgKLdLBST3LZZ6U8IB83LnCwKa7CX87OPxZVylK8W8NlzsjYt3P+ZOX8qUmhTxxkJa4i43aZXbQwv1a6QAwHY6n2/8mKdLpov5gf6Y28HpUN1d3mp3gegNK5quTNAxS2MzUXk9gOZ8ZXYLXCYjg4eWXXXCLxK+rmojXgsjdY3WruVLR+1YkXX38jJ0pirCp10knC+QGf1QTQE2quzYQNh8NQA+1weNUWi6xYd8c9Nt/bBdtoL56hqivJeNYiFw2PLeoGqfOtI1OrwR1Cp0WdoJ+cIdW9P/7ov already exist for user opnfv.
     Started: 18:22:50.336252
    Duration: 219.549 ms
     Changes:   

Summary for mas01.mcp-odl-noha.local
-------------
Succeeded: 32 (changed=11)
Failed:     0
Warnings:   2
-------------
Total states run:     32
Total run time:   51.701 s
[wait_for] OK: salt -C 'mas01*' state.apply maas.region
mas01.mcp-odl-noha.local:
----------
          ID: maas_login_admin
    Function: cmd.run
        Name: maas-region apikey --username opnfv > /var/lib/maas/.maas_credentials
      Result: True
     Comment: Command "maas-region apikey --username opnfv > /var/lib/maas/.maas_credentials" run
     Started: 18:23:01.154208
    Duration: 3880.327 ms
     Changes:   
              ----------
              pid:
                  19757
              retcode:
                  0
              stderr:
              stdout:
----------
          ID: create__machines
    Function: module.run
        Name: maas.process_machines
      Result: True
     Comment: Module function maas.process_machines executed
     Started: 18:23:05.037549
    Duration: 9655.64 ms
     Changes:   
              ----------
              ret:
                  ----------
                  errors:
                      ----------
                  success:
                  updated:
                      - cmp002
                      - cmp003
                      - cmp001
                      - odl01
                      - gtw01
                      - ctl01

Summary for mas01.mcp-odl-noha.local
------------
Succeeded: 2 (changed=2)
Failed:    0
------------
Total states run:     2
Total run time:  13.536 s
[NOTE] MaaS operations might take a long time, please be patient
tput: unknown terminal "unknown"
tput: unknown terminal "unknown"
ERROR: Minions returned with non-zero exit code
mas01.mcp-odl-noha.local:
----------
          ID: maas_login_admin
    Function: cmd.run
        Name: maas-region apikey --username opnfv > /var/lib/maas/.maas_credentials
      Result: True
     Comment: Command "maas-region apikey --username opnfv > /var/lib/maas/.maas_credentials" run
     Started: 18:23:54.755029
    Duration: 3998.832 ms
     Changes:   
              ----------
              pid:
                  20095
              retcode:
                  0
              stderr:
              stdout:
----------
          ID: wait_for_machines_ready_or_deployed
    Function: module.run
        Name: maas.wait_for_machine_status
      Result: False
     Comment: Module function maas.wait_for_machine_status threw an exception. Exception: Machines:['cmp002', 'cmp003', 'cmp001', 'odl01', 'gtw01', 'ctl01']not in Ready|Deployed state
     Started: 18:23:58.782515
    Duration: 1508192.511 ms
     Changes:   

Summary for mas01.mcp-odl-noha.local
------------
Succeeded: 1 (changed=1)
Failed:    1
------------
Total states run:     2
Total run time: 1512.191 s
[wait_for] Attempt 4/5 for: ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i /var/lib/opnfv/mcp.rsa ubuntu@10.20.0.2 sudo             CI_DEBUG=false ERASE_ENV=0             /root/fuel/mcp/config/states/maas
Warning: Permanently added '10.20.0.2' (ECDSA) to the list of known hosts.

[wait_for] Waiting for cmd to return success: salt --out yaml -C 'mas01*' service.status maas-fixup | fgrep -q 'false'
[wait_for] Attempt 1/60 for: salt --out yaml -C 'mas01*' service.status maas-fixup | fgrep -q 'false'
[wait_for] OK: salt --out yaml -C 'mas01*' service.status maas-fixup | fgrep -q 'false'

[wait_for] Waiting for cmd to return success: salt -C 'mas01*' state.apply salt,iptables
[wait_for] Attempt 1/10 for: salt -C 'mas01*' state.apply salt,iptables
mas01.mcp-odl-noha.local:
----------
          ID: salt_minion_packages
    Function: pkg.installed
        Name: salt-minion
      Result: True
     Comment: All specified packages are already installed
     Started: 18:49:45.241893
    Duration: 233.491 ms
     Changes:   
----------
          ID: salt_minion_dependency_packages
    Function: pkg.installed
      Result: True
     Comment: All specified packages are already installed
     Started: 18:49:45.475898
    Duration: 26.342 ms
     Changes:   
----------
          ID: /etc/salt/minion.d/minion.conf
    Function: file.managed
      Result: True
     Comment: File /etc/salt/minion.d/minion.conf is in the correct state
     Started: 18:49:45.509396
    Duration: 676.822 ms
     Changes:   
----------
          ID: salt_minion_systemd_override
    Function: file.managed
        Name: /etc/systemd/system/salt-minion.service.d/50-restarts.conf
      Result: True
     Comment: File /etc/systemd/system/salt-minion.service.d/50-restarts.conf is in the correct state
     Started: 18:49:46.192931
    Duration: 51.749 ms
     Changes:   
----------
          ID: salt_minion_systemd_reload
    Function: module.wait
        Name: service.systemctl_reload
      Result: True
     Comment: State was not run because none of the onchanges reqs changed
     Started: 18:49:46.246212
    Duration: 0.024 ms
     Changes:   
----------
          ID: salt_minion_service
    Function: service.running
        Name: salt-minion
      Result: True
     Comment: The service salt-minion is already running
     Started: 18:49:46.247043
    Duration: 301.106 ms
     Changes:   
----------
          ID: salt_minion_sync_all
    Function: module.run
        Name: saltutil.sync_all
      Result: True
     Comment: State was not run because none of the onchanges reqs changed
     Started: 18:49:46.550602
    Duration: 0.035 ms
     Changes:   
----------
          ID: salt_minion_grains_dir
    Function: file.directory
        Name: /etc/salt/grains.d
      Result: True
     Comment: Directory /etc/salt/grains.d is in the correct state
              Directory /etc/salt/grains.d updated
     Started: 18:49:46.550979
    Duration: 36.079 ms
     Changes:   
----------
          ID: salt_minion_grains_files
    Function: file.managed
        Name: /etc/salt/grains
      Result: True
     Comment: File /etc/salt/grains exists with proper permissions. No changes made.
     Started: 18:49:46.588124
    Duration: 1.611 ms
     Changes:   
----------
          ID: salt_minion_grains_files
    Function: file.managed
        Name: /etc/salt/grains.d/placeholder
      Result: True
     Comment: File /etc/salt/grains.d/placeholder exists with proper permissions. No changes made.
     Started: 18:49:46.590377
    Duration: 1.339 ms
     Changes:   
----------
          ID: salt_minion_grain_linux_sphinx
    Function: file.managed
        Name: /etc/salt/grains.d/sphinx
      Result: True
     Comment: File /etc/salt/grains.d/sphinx is in the correct state
     Started: 18:49:46.592325
    Duration: 22.702 ms
     Changes:   
----------
          ID: salt_minion_grain_linux_sphinx_validity_check
    Function: cmd.wait
        Name: python -c "import yaml; stream = file('/etc/salt/grains.d/sphinx', 'r'); yaml.load(stream); stream.close()"
      Result: True
     Comment: 
     Started: 18:49:46.618981
    Duration: 1.39 ms
     Changes:   
----------
          ID: salt_minion_grain_linux_dns_records
    Function: file.managed
        Name: /etc/salt/grains.d/dns_records
      Result: True
     Comment: File /etc/salt/grains.d/dns_records is in the correct state
     Started: 18:49:46.621071
    Duration: 25.783 ms
     Changes:   
----------
          ID: salt_minion_grain_linux_dns_records_validity_check
    Function: cmd.wait
        Name: python -c "import yaml; stream = file('/etc/salt/grains.d/dns_records', 'r'); yaml.load(stream); stream.close()"
      Result: True
     Comment: 
     Started: 18:49:46.648302
    Duration: 1.311 ms
     Changes:   
----------
          ID: salt_minion_grain_salt_salt
    Function: file.managed
        Name: /etc/salt/grains.d/salt
      Result: True
     Comment: File /etc/salt/grains.d/salt is in the correct state
     Started: 18:49:46.650249
    Duration: 28.481 ms
     Changes:   
----------
          ID: salt_minion_grain_salt_salt_validity_check
    Function: cmd.wait
        Name: python -c "import yaml; stream = file('/etc/salt/grains.d/salt', 'r'); yaml.load(stream); stream.close()"
      Result: True
     Comment: 
     Started: 18:49:46.680079
    Duration: 1.246 ms
     Changes:   
----------
          ID: salt_minion_grains_file
    Function: cmd.wait
        Name: cat /etc/salt/grains.d/* > /etc/salt/grains
      Result: True
     Comment: 
     Started: 18:49:46.683944
    Duration: 1.539 ms
     Changes:   
----------
          ID: salt_minion_grains_publish
    Function: module.wait
        Name: mine.update
      Result: True
     Comment: 
     Started: 18:49:46.686461
    Duration: 0.979 ms
     Changes:   
----------
          ID: salt_ca_certificates_packages
    Function: pkg.installed
        Name: ca-certificates
      Result: True
     Comment: All specified packages are already installed
     Started: 18:49:46.687770
    Duration: 40.856 ms
     Changes:   
----------
          ID: salt_update_certificates
    Function: cmd.wait
        Name: update-ca-certificates
      Result: True
     Comment: 
     Started: 18:49:46.731441
    Duration: 8.742 ms
     Changes:   
----------
          ID: iptables_packages_v4
    Function: pkg.installed
        Name: iptables
      Result: True
     Comment: All specified packages are already installed
     Started: 18:49:46.741382
    Duration: 16.663 ms
     Changes:   
----------
          ID: iptables_packages_v4
    Function: pkg.installed
        Name: iptables-persistent
      Result: True
     Comment: All specified packages are already installed
     Started: 18:49:46.759323
    Duration: 21.5 ms
     Changes:   
----------
          ID: iptables_modules_v4_load
    Function: kmod.present
      Result: True
     Comment: Kernel modules iptable_filter, ip_tables are already present
     Started: 18:49:46.783600
    Duration: 64.65 ms
     Changes:   
----------
          ID: /etc/iptables/rules.v4
    Function: file.managed
      Result: True
     Comment: File /etc/iptables/rules.v4 is in the correct state
     Started: 18:49:46.850180
    Duration: 217.593 ms
     Changes:   
----------
          ID: iptables_services_v4_start
    Function: cmd.run
        Name: find /usr/share/netfilter-persistent/plugins.d/[0-9]*-ip4tables -exec {} start \;
      Result: True
     Comment: onlyif execution failed
     Started: 18:49:47.070489
    Duration: 73.351 ms
     Changes:   
----------
          ID: netfilter-persistent
    Function: service.running
      Result: True
     Comment: The service netfilter-persistent is already running
     Started: 18:49:47.146357
    Duration: 192.981 ms
     Changes:   
----------
          ID: iptables_tables_cleanup_v4
    Function: module.wait
        Name: iptables_extra.remove_stale_tables
      Result: True
     Comment: 
     Started: 18:49:47.341520
    Duration: 3.837 ms
     Changes:   
----------
          ID: iptables_services_v6_stop
    Function: cmd.run
        Name: find /usr/share/netfilter-persistent/plugins.d/[0-9]*-ip6tables -exec {} flush \;
      Result: True
     Comment: onlyif execution failed
     Started: 18:49:47.346371
    Duration: 64.452 ms
     Changes:   
----------
          ID: /etc/iptables/rules.v6
    Function: file.absent
      Result: True
     Comment: File /etc/iptables/rules.v6 is not present
     Started: 18:49:47.412628
    Duration: 1.753 ms
     Changes:   
----------
          ID: iptables_tables_flush_v6
    Function: module.wait
        Name: iptables_extra.flush_all
      Result: True
     Comment: 
     Started: 18:49:47.415542
    Duration: 1.21 ms
     Changes:   
----------
          ID: salt_minion_service_restart
    Function: cmd.run
        Name: while true; do salt-call saltutil.running|grep fun: && continue; salt-call --local service.restart salt-minion; break; done
      Result: True
     Comment: State was not run because none of the onchanges reqs changed
     Started: 18:49:47.418180
    Duration: 0.027 ms
     Changes:   

Summary for mas01.mcp-odl-noha.local
-------------
Succeeded: 31
Failed:     0
-------------
Total states run:     31
Total run time:    2.120 s
[wait_for] OK: salt -C 'mas01*' state.apply salt,iptables
mas01.mcp-odl-noha.local:
----------
          ID: maas_cluster_packages
    Function: pkg.installed
        Name: maas-rack-controller
      Result: True
     Comment: All specified packages are already installed
     Started: 18:50:02.271890
    Duration: 295.715 ms
     Changes:   
----------
          ID: maas_cluster_packages
    Function: pkg.installed
        Name: ipmitool
      Result: True
     Comment: All specified packages are already installed
     Started: 18:50:02.568089
    Duration: 24.823 ms
     Changes:   
----------
          ID: /etc/maas/rackd.conf
    Function: file.line
      Result: True
     Comment: No changes needed to be made
     Started: 18:50:02.600091
    Duration: 2.656 ms
     Changes:   
----------
          ID: /etc/maas/rackd_conf
    Function: file.managed
        Name: /etc/maas/rackd.conf
      Result: True
     Comment: File /etc/maas/rackd.conf exists with proper permissions. No changes made.
     Started: 18:50:02.603117
    Duration: 2.576 ms
     Changes:   
----------
          ID: maas_cluster_services
    Function: service.running
        Name: maas-rackd
      Result: True
     Comment: The service maas-rackd is already running
     Started: 18:50:02.607456
    Duration: 343.84 ms
     Changes:   

Summary for mas01.mcp-odl-noha.local
------------
Succeeded: 5
Failed:    0
------------
Total states run:     5
Total run time: 669.610 ms

[wait_for] Waiting for cmd to return success: salt -C 'mas01*' state.apply maas.region
[wait_for] Attempt 1/10 for: salt -C 'mas01*' state.apply maas.region
mas01.mcp-odl-noha.local:
----------
          ID: maas_region_packages
    Function: pkg.installed
        Name: maas-region-controller
      Result: True
     Comment: All specified packages are already installed
     Started: 18:50:09.901783
    Duration: 231.771 ms
     Changes:   
----------
          ID: maas_region_packages
    Function: pkg.installed
        Name: python-oauth
      Result: True
     Comment: All specified packages are already installed
     Started: 18:50:10.134080
    Duration: 37.515 ms
     Changes:   
----------
          ID: /etc/maas/regiond.conf
    Function: file.replace
      Result: True
     Comment: No changes needed to be made
     Started: 18:50:10.201382
    Duration: 10.058 ms
     Changes:   
----------
          ID: /usr/lib/python3/dist-packages/provisioningserver/templates/proxy/maas-proxy.conf.template
    Function: file.managed
      Result: True
     Comment: File /usr/lib/python3/dist-packages/provisioningserver/templates/proxy/maas-proxy.conf.template is in the correct state
     Started: 18:50:10.212169
    Duration: 347.05 ms
     Changes:   
----------
          ID: maas_timeout_commissioning
    Function: file.replace
        Name: /usr/lib/python3/dist-packages/maasserver/node_status.py
      Result: True
     Comment: No changes needed to be made
     Started: 18:50:10.560005
    Duration: 6.827 ms
     Changes:   
----------
          ID: maas_timeout_deploying
    Function: file.replace
        Name: /usr/lib/python3/dist-packages/maasserver/node_status.py
      Result: True
     Comment: No changes needed to be made
     Started: 18:50:10.567545
    Duration: 10.597 ms
     Changes:   
----------
          ID: maas_interface_default_mode_dhcp
    Function: file.replace
        Name: /usr/lib/python3/dist-packages/maasserver/models/node.py
      Result: True
     Comment: No changes needed to be made
     Started: 18:50:10.578858
    Duration: 141.327 ms
     Changes:   
----------
          ID: /etc/apache2/conf-enabled/maas-http.conf
    Function: file.managed
      Result: True
     Comment: File /etc/apache2/conf-enabled/maas-http.conf is in the correct state
     Started: 18:50:10.720911
    Duration: 151.68 ms
     Changes:   
----------
          ID: maas_apache_headers
    Function: cmd.run
        Name: a2enmod headers
      Result: True
     Comment: Command "a2enmod headers" run
     Started: 18:50:10.874770
    Duration: 267.756 ms
     Changes:   
              ----------
              pid:
                  26957
              retcode:
                  0
              stderr:
              stdout:
                  Module headers already enabled
----------
          ID: /usr/share/maas/web/static/css/maas-styles.css
    Function: file.managed
      Result: True
     Comment: File /usr/share/maas/web/static/css/maas-styles.css is in the correct state
     Started: 18:50:11.143057
    Duration: 155.144 ms
     Changes:   
----------
          ID: /etc/maas/preseeds/curtin_userdata_amd64_generic_trusty
    Function: file.managed
      Result: True
     Comment: File /etc/maas/preseeds/curtin_userdata_amd64_generic_trusty is in the correct state
     Started: 18:50:11.299086
    Duration: 338.736 ms
     Changes:   
----------
          ID: /etc/maas/preseeds/curtin_userdata_amd64_generic_xenial
    Function: file.managed
      Result: True
     Comment: File /etc/maas/preseeds/curtin_userdata_amd64_generic_xenial is in the correct state
     Started: 18:50:11.638560
    Duration: 253.17 ms
     Changes:   
----------
          ID: /etc/maas/preseeds/curtin_userdata_arm64_generic_xenial
    Function: file.managed
      Result: True
     Comment: File /etc/maas/preseeds/curtin_userdata_arm64_generic_xenial is in the correct state
     Started: 18:50:11.892473
    Duration: 263.557 ms
     Changes:   
----------
          ID: Configure /root/.pgpass for MAAS
    Function: file.managed
        Name: /root/.pgpass
      Result: True
     Comment: File /root/.pgpass is in the correct state
     Started: 18:50:12.163392
    Duration: 100.833 ms
     Changes:   
----------
          ID: maas_region_syncdb
    Function: cmd.run
        Name: maas-region syncdb --noinput
      Result: True
     Comment: Command "maas-region syncdb --noinput" run
     Started: 18:50:12.271712
    Duration: 7230.025 ms
     Changes:   
              ----------
              pid:
                  26971
              retcode:
                  0
              stderr:
              stdout:
                  Operations to perform:
                    Synchronize unmigrated apps: staticfiles, messages
                    Apply all migrations: auth, contenttypes, piston3, metadataserver, sites, sessions, maasserver
                  Synchronizing apps without migrations:
                    Creating tables...
                      Running deferred SQL...
                    Installing custom SQL...
                  Running migrations:
                    No migrations to apply.
----------
          ID: maas_region_services
    Function: service.running
        Name: maas-regiond
      Result: True
     Comment: The service maas-regiond is already running
     Started: 18:50:19.521519
    Duration: 197.817 ms
     Changes:   
----------
          ID: maas_region_services
    Function: service.running
        Name: bind9
      Result: True
     Comment: The service bind9 is already running
     Started: 18:50:19.721793
    Duration: 178.406 ms
     Changes:   
----------
          ID: maas_region_services
    Function: service.running
        Name: apache2
      Result: True
     Comment: The service apache2 is already running
     Started: 18:50:19.903809
    Duration: 202.663 ms
     Changes:   
----------
          ID: maas_warmup
    Function: module.run
        Name: maasng.wait_for_http_code
      Result: True
     Comment: Module function maasng.wait_for_http_code executed
     Started: 18:50:20.110646
    Duration: 63.921 ms
     Changes:   
              ----------
              ret:
                  ----------
                  comment:
                      MAAS API:http://localhost:5240/MAAS up.
                  result:
                      True
----------
          ID: maas_set_admin_password
    Function: cmd.run
        Name: maas createadmin --username opnfv --password opnfv_secret --email email@example.com && touch /var/lib/maas/.setup_admin
      Result: True
     Comment: /var/lib/maas/.setup_admin exists
     Started: 18:50:20.176450
    Duration: 3.41 ms
     Changes:   
----------
          ID: maas_login_admin
    Function: cmd.run
        Name: maas-region apikey --username opnfv > /var/lib/maas/.maas_credentials
      Result: True
     Comment: Command "maas-region apikey --username opnfv > /var/lib/maas/.maas_credentials" run
     Started: 18:50:20.200941
    Duration: 5853.974 ms
     Changes:   
              ----------
              pid:
                  26990
              retcode:
                  0
              stderr:
              stdout:
----------
          ID: maas_region_boot_source_resources_mirror
    Function: maasng.boot_source_present
      Result: True
     Comment: boot-source http://images.maas.io/ephemeral-v3/daily alredy exist
     Started: 18:50:26.090403
    Duration: 574.743 ms
     Changes:   
              ----------
              changes:
                  ----------
    Warnings: 'name' is an invalid keyword argument for
              'maasng.boot_source_present'. If you were trying to pass
              additional data to be used in a template context, please populate
              'context' with 'key: value' pairs. Your approach will work until
              Salt Fluorine is out. Please update your state files.
----------
          ID: maas_wait_for_region_import_done
    Function: module.run
        Name: maasng.boot_resources_import
      Result: True
     Comment: Module function maasng.boot_resources_import executed
     Started: 18:50:26.666811
    Duration: 26304.84 ms
     Changes:   
              ----------
              ret:
                  True
----------
          ID: maas_region_boot_sources_selection_xenial
    Function: maasng.boot_sources_selections_present
      Result: True
     Comment: Requested boot-source selection for http://images.maas.io/ephemeral-v3/daily already exist.
     Started: 18:50:52.973389
    Duration: 539.274 ms
     Changes:   
    Warnings: 'name' is an invalid keyword argument for
              'maasng.boot_sources_selections_present'. If you were trying to
              pass additional data to be used in a template context, please
              populate 'context' with 'key: value' pairs. Your approach will
              work until Salt Fluorine is out. Please update your state files.
----------
          ID: maas_wait_for_racks_import_done
    Function: module.run
        Name: maasng.sync_and_wait_bs_to_all_racks
      Result: True
     Comment: Module function maasng.sync_and_wait_bs_to_all_racks executed
     Started: 18:50:53.513963
    Duration: 3911.864 ms
     Changes:   
              ----------
              ret:
                  True
----------
          ID: maas_config
    Function: module.run
        Name: maas.process_maas_config
      Result: True
     Comment: Module function maas.process_maas_config executed
     Started: 18:50:57.427932
    Duration: 7802.106 ms
     Changes:   
              ----------
              ret:
                  ----------
                  errors:
                      ----------
                  success:
                      - enable_http_proxy
                      - upstream_dns
                      - commissioning_distro_series
                      - default_osystem
                      - active_discovery_interval
                      - dnssec_validation
                      - maas_name
                      - network_discovery
                      - enable_third_party_drivers
                      - default_storage_layout
                      - ntp_external_only
                      - disk_erase_with_secure_erase
                      - default_distro_series
                      - default_min_hwe_kernel
                  updated:
----------
          ID: maas_fabrics_pxe_admin
    Function: maasng.fabric_present
        Name: pxe_admin
      Result: True
     Comment: Module function maasng.fabric_present executed
     Started: 18:51:05.231556
    Duration: 1035.668 ms
     Changes:   
              ----------
              new:
                  Fabric  pxe_admin created
              result:
                  True
----------
          ID: maas_vlan0_present_for_pxe_admin
    Function: maasng.vlan_present_in_fabric
        Name: vlan 0
      Result: True
     Comment: Vlan vlan 0 has been updated for pxe_admin
     Started: 18:51:06.268146
    Duration: 2075.133 ms
     Changes:   
              ----------
              new:
                  Vlan untagged was updated
----------
          ID: maas_create_subnet_10.1.0.0/24
    Function: maasng.subnet_present
        Name: 10.1.0.0/24
      Result: True
     Comment: Subnet 10.1.0.0/24 has been updated for pxe_admin
     Started: 18:51:08.346598
    Duration: 1924.51 ms
     Changes:   
----------
          ID: maas_create_iprange_1
    Function: maasng.iprange_present
      Result: True
     Comment: Iprange maas_create_iprange_1 already exist.
     Started: 18:51:10.272237
    Duration: 258.97 ms
     Changes:   
----------
          ID: maas_vlan0_present_for_pxe_admin_dhcp
    Function: maasng.vlan_present_in_fabric
        Name: vlan 0
      Result: True
     Comment: Vlan vlan 0 has been updated for pxe_admin
     Started: 18:51:10.531640
    Duration: 2339.018 ms
     Changes:   
              ----------
              new:
                  Vlan untagged was updated
----------
          ID: maas_sshkey_1
    Function: maasng.sshkey_present
        Name: opnfv
      Result: True
     Comment: SSH key ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9iteXSOD1Du/fG2coTPKPKmlyjD5EHa0wwZm9pdya5BuWatdcIzkCgKLdLBST3LZZ6U8IB83LnCwKa7CX87OPxZVylK8W8NlzsjYt3P+ZOX8qUmhTxxkJa4i43aZXbQwv1a6QAwHY6n2/8mKdLpov5gf6Y28HpUN1d3mp3gegNK5quTNAxS2MzUXk9gOZ8ZXYLXCYjg4eWXXXCLxK+rmojXgsjdY3WruVLR+1YkXX38jJ0pirCp10knC+QGf1QTQE2quzYQNh8NQA+1weNUWi6xYd8c9Nt/bBdtoL56hqivJeNYiFw2PLeoGqfOtI1OrwR1Cp0WdoJ+cIdW9P/7ov already exist for user opnfv.
     Started: 18:51:12.871454
    Duration: 498.986 ms
     Changes:   

Summary for mas01.mcp-odl-noha.local
-------------
Succeeded: 32 (changed=11)
Failed:     0
Warnings:   2
-------------
Total states run:     32
Total run time:   63.311 s
[wait_for] OK: salt -C 'mas01*' state.apply maas.region
mas01.mcp-odl-noha.local:
----------
          ID: maas_login_admin
    Function: cmd.run
        Name: maas-region apikey --username opnfv > /var/lib/maas/.maas_credentials
      Result: True
     Comment: Command "maas-region apikey --username opnfv > /var/lib/maas/.maas_credentials" run
     Started: 18:51:26.389154
    Duration: 5763.144 ms
     Changes:   
              ----------
              pid:
                  27455
              retcode:
                  0
              stderr:
              stdout:
----------
          ID: create__machines
    Function: module.run
        Name: maas.process_machines
      Result: True
     Comment: Module function maas.process_machines executed
     Started: 18:51:32.163151
    Duration: 18468.925 ms
     Changes:   
              ----------
              ret:
                  ----------
                  errors:
                      ----------
                  success:
                  updated:
                      - cmp002
                      - cmp003
                      - cmp001
                      - odl01
                      - gtw01
                      - ctl01

Summary for mas01.mcp-odl-noha.local
------------
Succeeded: 2 (changed=2)
Failed:    0
------------
Total states run:     2
Total run time:  24.232 s
[NOTE] MaaS operations might take a long time, please be patient
tput: unknown terminal "unknown"
tput: unknown terminal "unknown"
mas01.mcp-odl-noha.local:
----------
          ID: maas_login_admin
    Function: cmd.run
        Name: maas-region apikey --username opnfv > /var/lib/maas/.maas_credentials
      Result: True
     Comment: Command "maas-region apikey --username opnfv > /var/lib/maas/.maas_credentials" run
     Started: 18:52:36.089110
    Duration: 6215.895 ms
     Changes:   
              ----------
              pid:
                  27821
              retcode:
                  0
              stderr:
              stdout:
----------
          ID: wait_for_machines_ready_or_deployed
    Function: module.run
        Name: maas.wait_for_machine_status
      Result: False
     Comment: Module function maas.wait_for_machine_status threw an exception. Exception: Machines:['cmp002', 'cmp003', 'cmp001', 'odl01', 'gtw01', 'ctl01']not in Ready|Deployed state
     Started: 18:52:42.309778
    Duration: 1517002.304 ms
     Changes:   

Summary for mas01.mcp-odl-noha.local
------------
Succeeded: 1 (changed=1)
Failed:    1
------------
Total states run:     2
Total run time: 1523.218 s
ERROR: Minions returned with non-zero exit code
[wait_for] Attempt 5/5 for: ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i /var/lib/opnfv/mcp.rsa ubuntu@10.20.0.2 sudo             CI_DEBUG=false ERASE_ENV=0             /root/fuel/mcp/config/states/maas
Warning: Permanently added '10.20.0.2' (ECDSA) to the list of known hosts.

[wait_for] Waiting for cmd to return success: salt --out yaml -C 'mas01*' service.status maas-fixup | fgrep -q 'false'
[wait_for] Attempt 1/60 for: salt --out yaml -C 'mas01*' service.status maas-fixup | fgrep -q 'false'
[wait_for] OK: salt --out yaml -C 'mas01*' service.status maas-fixup | fgrep -q 'false'

[wait_for] Waiting for cmd to return success: salt -C 'mas01*' state.apply salt,iptables
[wait_for] Attempt 1/10 for: salt -C 'mas01*' state.apply salt,iptables
mas01.mcp-odl-noha.local:
----------
          ID: salt_minion_packages
    Function: pkg.installed
        Name: salt-minion
      Result: True
     Comment: All specified packages are already installed
     Started: 19:18:41.468806
    Duration: 201.334 ms
     Changes:   
----------
          ID: salt_minion_dependency_packages
    Function: pkg.installed
      Result: True
     Comment: All specified packages are already installed
     Started: 19:18:41.671257
    Duration: 17.388 ms
     Changes:   
----------
          ID: /etc/salt/minion.d/minion.conf
    Function: file.managed
      Result: True
     Comment: File /etc/salt/minion.d/minion.conf is in the correct state
     Started: 19:18:41.695511
    Duration: 602.042 ms
     Changes:   
----------
          ID: salt_minion_systemd_override
    Function: file.managed
        Name: /etc/systemd/system/salt-minion.service.d/50-restarts.conf
      Result: True
     Comment: File /etc/systemd/system/salt-minion.service.d/50-restarts.conf is in the correct state
     Started: 19:18:42.335719
    Duration: 64.457 ms
     Changes:   
----------
          ID: salt_minion_systemd_reload
    Function: module.wait
        Name: service.systemctl_reload
      Result: True
     Comment: State was not run because none of the onchanges reqs changed
     Started: 19:18:42.400907
    Duration: 0.021 ms
     Changes:   
----------
          ID: salt_minion_service
    Function: service.running
        Name: salt-minion
      Result: True
     Comment: The service salt-minion is already running
     Started: 19:18:42.402518
    Duration: 275.314 ms
     Changes:   
----------
          ID: salt_minion_sync_all
    Function: module.run
        Name: saltutil.sync_all
      Result: True
     Comment: State was not run because none of the onchanges reqs changed
     Started: 19:18:42.679834
    Duration: 0.044 ms
     Changes:   
----------
          ID: salt_minion_grains_dir
    Function: file.directory
        Name: /etc/salt/grains.d
      Result: True
     Comment: Directory /etc/salt/grains.d is in the correct state
              Directory /etc/salt/grains.d updated
     Started: 19:18:42.680236
    Duration: 2.498 ms
     Changes:   
----------
          ID: salt_minion_grains_files
    Function: file.managed
        Name: /etc/salt/grains
      Result: True
     Comment: File /etc/salt/grains exists with proper permissions. No changes made.
     Started: 19:18:42.683757
    Duration: 1.572 ms
     Changes:   
----------
          ID: salt_minion_grains_files
    Function: file.managed
        Name: /etc/salt/grains.d/placeholder
      Result: True
     Comment: File /etc/salt/grains.d/placeholder exists with proper permissions. No changes made.
     Started: 19:18:42.685963
    Duration: 1.386 ms
     Changes:   
----------
          ID: salt_minion_grain_linux_sphinx
    Function: file.managed
        Name: /etc/salt/grains.d/sphinx
      Result: True
     Comment: File /etc/salt/grains.d/sphinx is in the correct state
     Started: 19:18:42.687974
    Duration: 17.967 ms
     Changes:   
----------
          ID: salt_minion_grain_linux_sphinx_validity_check
    Function: cmd.wait
        Name: python -c "import yaml; stream = file('/etc/salt/grains.d/sphinx', 'r'); yaml.load(stream); stream.close()"
      Result: True
     Comment: 
     Started: 19:18:42.709789
    Duration: 1.355 ms
     Changes:   
----------
          ID: salt_minion_grain_linux_dns_records
    Function: file.managed
        Name: /etc/salt/grains.d/dns_records
      Result: True
     Comment: File /etc/salt/grains.d/dns_records is in the correct state
     Started: 19:18:42.711780
    Duration: 14.08 ms
     Changes:   
----------
          ID: salt_minion_grain_linux_dns_records_validity_check
    Function: cmd.wait
        Name: python -c "import yaml; stream = file('/etc/salt/grains.d/dns_records', 'r'); yaml.load(stream); stream.close()"
      Result: True
     Comment: 
     Started: 19:18:42.727317
    Duration: 1.351 ms
     Changes:   
----------
          ID: salt_minion_grain_salt_salt
    Function: file.managed
        Name: /etc/salt/grains.d/salt
      Result: True
     Comment: File /etc/salt/grains.d/salt is in the correct state
     Started: 19:18:42.729332
    Duration: 2.38 ms
     Changes:   
----------
          ID: salt_minion_grain_salt_salt_validity_check
    Function: cmd.wait
        Name: python -c "import yaml; stream = file('/etc/salt/grains.d/salt', 'r'); yaml.load(stream); stream.close()"
      Result: True
     Comment: 
     Started: 19:18:42.733070
    Duration: 1.169 ms
     Changes:   
----------
          ID: salt_minion_grains_file
    Function: cmd.wait
        Name: cat /etc/salt/grains.d/* > /etc/salt/grains
      Result: True
     Comment: 
     Started: 19:18:42.736878
    Duration: 1.529 ms
     Changes:   
----------
          ID: salt_minion_grains_publish
    Function: module.wait
        Name: mine.update
      Result: True
     Comment: 
     Started: 19:18:42.739396
    Duration: 0.973 ms
     Changes:   
----------
          ID: salt_ca_certificates_packages
    Function: pkg.installed
        Name: ca-certificates
      Result: True
     Comment: All specified packages are already installed
     Started: 19:18:42.740730
    Duration: 20.584 ms
     Changes:   
----------
          ID: salt_update_certificates
    Function: cmd.wait
        Name: update-ca-certificates
      Result: True
     Comment: 
     Started: 19:18:42.763167
    Duration: 1.277 ms
     Changes:   
----------
          ID: iptables_packages_v4
    Function: pkg.installed
        Name: iptables
      Result: True
     Comment: All specified packages are already installed
     Started: 19:18:42.764806
    Duration: 17.601 ms
     Changes:   
----------
          ID: iptables_packages_v4
    Function: pkg.installed
        Name: iptables-persistent
      Result: True
     Comment: All specified packages are already installed
     Started: 19:18:42.782799
    Duration: 13.213 ms
     Changes:   
----------
          ID: iptables_modules_v4_load
    Function: kmod.present
      Result: True
     Comment: Kernel modules iptable_filter, ip_tables are already present
     Started: 19:18:42.802331
    Duration: 39.221 ms
     Changes:   
----------
          ID: /etc/iptables/rules.v4
    Function: file.managed
      Result: True
     Comment: File /etc/iptables/rules.v4 is in the correct state
     Started: 19:18:42.842552
    Duration: 223.558 ms
     Changes:   
----------
          ID: iptables_services_v4_start
    Function: cmd.run
        Name: find /usr/share/netfilter-persistent/plugins.d/[0-9]*-ip4tables -exec {} start \;
      Result: True
     Comment: onlyif execution failed
     Started: 19:18:43.067816
    Duration: 102.217 ms
     Changes:   
----------
          ID: netfilter-persistent
    Function: service.running
      Result: True
     Comment: The service netfilter-persistent is already running
     Started: 19:18:43.171591
    Duration: 160.048 ms
     Changes:   
----------
          ID: iptables_tables_cleanup_v4
    Function: module.wait
        Name: iptables_extra.remove_stale_tables
      Result: True
     Comment: 
     Started: 19:18:43.333202
    Duration: 1.79 ms
     Changes:   
----------
          ID: iptables_services_v6_stop
    Function: cmd.run
        Name: find /usr/share/netfilter-persistent/plugins.d/[0-9]*-ip6tables -exec {} flush \;
      Result: True
     Comment: onlyif execution failed
     Started: 19:18:43.335345
    Duration: 31.404 ms
     Changes:   
----------
          ID: /etc/iptables/rules.v6
    Function: file.absent
      Result: True
     Comment: File /etc/iptables/rules.v6 is not present
     Started: 19:18:43.368272
    Duration: 1.663 ms
     Changes:   
----------
          ID: iptables_tables_flush_v6
    Function: module.wait
        Name: iptables_extra.flush_all
      Result: True
     Comment: 
     Started: 19:18:43.371031
    Duration: 1.121 ms
     Changes:   
----------
          ID: salt_minion_service_restart
    Function: cmd.run
        Name: while true; do salt-call saltutil.running|grep fun: && continue; salt-call --local service.restart salt-minion; break; done
      Result: True
     Comment: State was not run because none of the onchanges reqs changed
     Started: 19:18:43.373484
    Duration: 0.025 ms
     Changes:   

Summary for mas01.mcp-odl-noha.local
-------------
Succeeded: 31
Failed:     0
-------------
Total states run:     31
Total run time:    1.821 s
[wait_for] OK: salt -C 'mas01*' state.apply salt,iptables
mas01.mcp-odl-noha.local:
----------
          ID: maas_cluster_packages
    Function: pkg.installed
        Name: maas-rack-controller
      Result: True
     Comment: All specified packages are already installed
     Started: 19:18:57.491337
    Duration: 230.663 ms
     Changes:   
----------
          ID: maas_cluster_packages
    Function: pkg.installed
        Name: ipmitool
      Result: True
     Comment: All specified packages are already installed
     Started: 19:18:57.722537
    Duration: 18.547 ms
     Changes:   
----------
          ID: /etc/maas/rackd.conf
    Function: file.line
      Result: True
     Comment: No changes needed to be made
     Started: 19:18:57.747376
    Duration: 2.723 ms
     Changes:   
----------
          ID: /etc/maas/rackd_conf
    Function: file.managed
        Name: /etc/maas/rackd.conf
      Result: True
     Comment: File /etc/maas/rackd.conf exists with proper permissions. No changes made.
     Started: 19:18:57.750466
    Duration: 3.008 ms
     Changes:   
----------
          ID: maas_cluster_services
    Function: service.running
        Name: maas-rackd
      Result: True
     Comment: The service maas-rackd is already running
     Started: 19:18:57.755296
    Duration: 255.138 ms
     Changes:   

Summary for mas01.mcp-odl-noha.local
------------
Succeeded: 5
Failed:    0
------------
Total states run:     5
Total run time: 510.079 ms

[wait_for] Waiting for cmd to return success: salt -C 'mas01*' state.apply maas.region
[wait_for] Attempt 1/10 for: salt -C 'mas01*' state.apply maas.region
mas01.mcp-odl-noha.local:
----------
          ID: maas_region_packages
    Function: pkg.installed
        Name: maas-region-controller
      Result: True
     Comment: All specified packages are already installed
     Started: 19:19:04.607814
    Duration: 213.428 ms
     Changes:   
----------
          ID: maas_region_packages
    Function: pkg.installed
        Name: python-oauth
      Result: True
     Comment: All specified packages are already installed
     Started: 19:19:04.821698
    Duration: 13.269 ms
     Changes:   
----------
          ID: /etc/maas/regiond.conf
    Function: file.replace
      Result: True
     Comment: No changes needed to be made
     Started: 19:19:04.845003
    Duration: 11.327 ms
     Changes:   
----------
          ID: /usr/lib/python3/dist-packages/provisioningserver/templates/proxy/maas-proxy.conf.template
    Function: file.managed
      Result: True
     Comment: File /usr/lib/python3/dist-packages/provisioningserver/templates/proxy/maas-proxy.conf.template is in the correct state
     Started: 19:19:04.857114
    Duration: 241.137 ms
     Changes:   
----------
          ID: maas_timeout_commissioning
    Function: file.replace
        Name: /usr/lib/python3/dist-packages/maasserver/node_status.py
      Result: True
     Comment: No changes needed to be made
     Started: 19:19:05.098990
    Duration: 9.176 ms
     Changes:   
----------
          ID: maas_timeout_deploying
    Function: file.replace
        Name: /usr/lib/python3/dist-packages/maasserver/node_status.py
      Result: True
     Comment: No changes needed to be made
     Started: 19:19:05.108893
    Duration: 6.187 ms
     Changes:   
----------
          ID: maas_interface_default_mode_dhcp
    Function: file.replace
        Name: /usr/lib/python3/dist-packages/maasserver/models/node.py
      Result: True
     Comment: No changes needed to be made
     Started: 19:19:05.115787
    Duration: 53.563 ms
     Changes:   
----------
          ID: /etc/apache2/conf-enabled/maas-http.conf
    Function: file.managed
      Result: True
     Comment: File /etc/apache2/conf-enabled/maas-http.conf is in the correct state
     Started: 19:19:05.170067
    Duration: 121.872 ms
     Changes:   
----------
          ID: maas_apache_headers
    Function: cmd.run
        Name: a2enmod headers
      Result: True
     Comment: Command "a2enmod headers" run
     Started: 19:19:05.295144
    Duration: 215.205 ms
     Changes:   
              ----------
              pid:
                  1326
              retcode:
                  0
              stderr:
              stdout:
                  Module headers already enabled
----------
          ID: /usr/share/maas/web/static/css/maas-styles.css
    Function: file.managed
      Result: True
     Comment: File /usr/share/maas/web/static/css/maas-styles.css is in the correct state
     Started: 19:19:05.510875
    Duration: 141.609 ms
     Changes:   
----------
          ID: /etc/maas/preseeds/curtin_userdata_amd64_generic_trusty
    Function: file.managed
      Result: True
     Comment: File /etc/maas/preseeds/curtin_userdata_amd64_generic_trusty is in the correct state
     Started: 19:19:05.653332
    Duration: 219.567 ms
     Changes:   
----------
          ID: /etc/maas/preseeds/curtin_userdata_amd64_generic_xenial
    Function: file.managed
      Result: True
     Comment: File /etc/maas/preseeds/curtin_userdata_amd64_generic_xenial is in the correct state
     Started: 19:19:05.873620
    Duration: 434.734 ms
     Changes:   
----------
          ID: /etc/maas/preseeds/curtin_userdata_arm64_generic_xenial
    Function: file.managed
      Result: True
     Comment: File /etc/maas/preseeds/curtin_userdata_arm64_generic_xenial is in the correct state
     Started: 19:19:06.313651
    Duration: 157.19 ms
     Changes:   
----------
          ID: Configure /root/.pgpass for MAAS
    Function: file.managed
        Name: /root/.pgpass
      Result: True
     Comment: File /root/.pgpass is in the correct state
     Started: 19:19:06.471694
    Duration: 257.16 ms
     Changes:   
----------
          ID: maas_region_syncdb
    Function: cmd.run
        Name: maas-region syncdb --noinput
      Result: True
     Comment: Command "maas-region syncdb --noinput" run
     Started: 19:19:06.736397
    Duration: 6768.618 ms
     Changes:   
              ----------
              pid:
                  1344
              retcode:
                  0
              stderr:
              stdout:
                  Operations to perform:
                    Synchronize unmigrated apps: messages, staticfiles
                    Apply all migrations: maasserver, metadataserver, contenttypes, piston3, auth, sites, sessions
                  Synchronizing apps without migrations:
                    Creating tables...
                      Running deferred SQL...
                    Installing custom SQL...
                  Running migrations:
                    No migrations to apply.
----------
          ID: maas_region_services
    Function: service.running
        Name: maas-regiond
      Result: True
     Comment: The service maas-regiond is already running
     Started: 19:19:13.507861
    Duration: 271.574 ms
     Changes:   
----------
          ID: maas_region_services
    Function: service.running
        Name: bind9
      Result: True
     Comment: The service bind9 is already running
     Started: 19:19:13.783072
    Duration: 200.092 ms
     Changes:   
----------
          ID: maas_region_services
    Function: service.running
        Name: apache2
      Result: True
     Comment: The service apache2 is already running
     Started: 19:19:13.986025
    Duration: 185.761 ms
     Changes:   
----------
          ID: maas_warmup
    Function: module.run
        Name: maasng.wait_for_http_code
      Result: True
     Comment: Module function maasng.wait_for_http_code executed
     Started: 19:19:14.174471
    Duration: 651.04 ms
     Changes:   
              ----------
              ret:
                  ----------
                  comment:
                      MAAS API:http://localhost:5240/MAAS up.
                  result:
                      True
----------
          ID: maas_set_admin_password
    Function: cmd.run
        Name: maas createadmin --username opnfv --password opnfv_secret --email email@example.com && touch /var/lib/maas/.setup_admin
      Result: True
     Comment: /var/lib/maas/.setup_admin exists
     Started: 19:19:14.826731
    Duration: 1.375 ms
     Changes:   
----------
          ID: maas_login_admin
    Function: cmd.run
        Name: maas-region apikey --username opnfv > /var/lib/maas/.maas_credentials
      Result: True
     Comment: Command "maas-region apikey --username opnfv > /var/lib/maas/.maas_credentials" run
     Started: 19:19:14.835680
    Duration: 5187.508 ms
     Changes:   
              ----------
              pid:
                  1375
              retcode:
                  0
              stderr:
              stdout:
----------
          ID: maas_region_boot_source_resources_mirror
    Function: maasng.boot_source_present
      Result: True
     Comment: boot-source http://images.maas.io/ephemeral-v3/daily alredy exist
     Started: 19:19:20.040021
    Duration: 696.034 ms
     Changes:   
              ----------
              changes:
                  ----------
    Warnings: 'name' is an invalid keyword argument for
              'maasng.boot_source_present'. If you were trying to pass
              additional data to be used in a template context, please populate
              'context' with 'key: value' pairs. Your approach will work until
              Salt Fluorine is out. Please update your state files.
----------
          ID: maas_wait_for_region_import_done
    Function: module.run
        Name: maasng.boot_resources_import
      Result: True
     Comment: Module function maasng.boot_resources_import executed
     Started: 19:19:20.738259
    Duration: 32354.383 ms
     Changes:   
              ----------
              ret:
                  True
----------
          ID: maas_region_boot_sources_selection_xenial
    Function: maasng.boot_sources_selections_present
      Result: True
     Comment: Requested boot-source selection for http://images.maas.io/ephemeral-v3/daily already exist.
     Started: 19:19:53.094334
    Duration: 376.072 ms
     Changes:   
    Warnings: 'name' is an invalid keyword argument for
              'maasng.boot_sources_selections_present'. If you were trying to
              pass additional data to be used in a template context, please
              populate 'context' with 'key: value' pairs. Your approach will
              work until Salt Fluorine is out. Please update your state files.
----------
          ID: maas_wait_for_racks_import_done
    Function: module.run
        Name: maasng.sync_and_wait_bs_to_all_racks
      Result: True
     Comment: Module function maasng.sync_and_wait_bs_to_all_racks executed
     Started: 19:19:53.471537
    Duration: 3163.863 ms
     Changes:   
              ----------
              ret:
                  True
----------
          ID: maas_config
    Function: module.run
        Name: maas.process_maas_config
      Result: True
     Comment: Module function maas.process_maas_config executed
     Started: 19:19:56.637290
    Duration: 10075.058 ms
     Changes:   
              ----------
              ret:
                  ----------
                  errors:
                      ----------
                  success:
                      - enable_http_proxy
                      - upstream_dns
                      - commissioning_distro_series
                      - default_osystem
                      - active_discovery_interval
                      - dnssec_validation
                      - maas_name
                      - network_discovery
                      - enable_third_party_drivers
                      - default_storage_layout
                      - ntp_external_only
                      - disk_erase_with_secure_erase
                      - default_distro_series
                      - default_min_hwe_kernel
                  updated:
----------
          ID: maas_fabrics_pxe_admin
    Function: maasng.fabric_present
        Name: pxe_admin
      Result: True
     Comment: Module function maasng.fabric_present executed
     Started: 19:20:06.713789
    Duration: 333.618 ms
     Changes:   
              ----------
              new:
                  Fabric  pxe_admin created
              result:
                  True
----------
          ID: maas_vlan0_present_for_pxe_admin
    Function: maasng.vlan_present_in_fabric
        Name: vlan 0
      Result: True
     Comment: Vlan vlan 0 has been updated for pxe_admin
     Started: 19:20:07.048159
    Duration: 2259.7 ms
     Changes:   
              ----------
              new:
                  Vlan untagged was updated
----------
          ID: maas_create_subnet_10.1.0.0/24
    Function: maasng.subnet_present
        Name: 10.1.0.0/24
      Result: True
     Comment: Subnet 10.1.0.0/24 has been updated for pxe_admin
     Started: 19:20:09.309865
    Duration: 1305.466 ms
     Changes:   
----------
          ID: maas_create_iprange_1
    Function: maasng.iprange_present
      Result: True
     Comment: Iprange maas_create_iprange_1 already exist.
     Started: 19:20:10.617019
    Duration: 310.334 ms
     Changes:   
----------
          ID: maas_vlan0_present_for_pxe_admin_dhcp
    Function: maasng.vlan_present_in_fabric
        Name: vlan 0
      Result: True
     Comment: Vlan vlan 0 has been updated for pxe_admin
     Started: 19:20:10.928177
    Duration: 1603.037 ms
     Changes:   
              ----------
              new:
                  Vlan untagged was updated
----------
          ID: maas_sshkey_1
    Function: maasng.sshkey_present
        Name: opnfv
      Result: True
     Comment: SSH key ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9iteXSOD1Du/fG2coTPKPKmlyjD5EHa0wwZm9pdya5BuWatdcIzkCgKLdLBST3LZZ6U8IB83LnCwKa7CX87OPxZVylK8W8NlzsjYt3P+ZOX8qUmhTxxkJa4i43aZXbQwv1a6QAwHY6n2/8mKdLpov5gf6Y28HpUN1d3mp3gegNK5quTNAxS2MzUXk9gOZ8ZXYLXCYjg4eWXXXCLxK+rmojXgsjdY3WruVLR+1YkXX38jJ0pirCp10knC+QGf1QTQE2quzYQNh8NQA+1weNUWi6xYd8c9Nt/bBdtoL56hqivJeNYiFw2PLeoGqfOtI1OrwR1Cp0WdoJ+cIdW9P/7ov already exist for user opnfv.
     Started: 19:20:12.532574
    Duration: 129.046 ms
     Changes:   

Summary for mas01.mcp-odl-noha.local
-------------
Succeeded: 32 (changed=11)
Failed:     0
Warnings:   2
-------------
Total states run:     32
Total run time:   67.968 s
[wait_for] OK: salt -C 'mas01*' state.apply maas.region
mas01.mcp-odl-noha.local:
----------
          ID: maas_login_admin
    Function: cmd.run
        Name: maas-region apikey --username opnfv > /var/lib/maas/.maas_credentials
      Result: True
     Comment: Command "maas-region apikey --username opnfv > /var/lib/maas/.maas_credentials" run
     Started: 19:20:26.591525
    Duration: 6798.936 ms
     Changes:   
              ----------
              pid:
                  1853
              retcode:
                  0
              stderr:
              stdout:
----------
          ID: create__machines
    Function: module.run
        Name: maas.process_machines
      Result: True
     Comment: Module function maas.process_machines executed
     Started: 19:20:33.413413
    Duration: 18270.558 ms
     Changes:   
              ----------
              ret:
                  ----------
                  errors:
                      ----------
                  success:
                  updated:
                      - cmp002
                      - cmp003
                      - cmp001
                      - odl01
                      - gtw01
                      - ctl01

Summary for mas01.mcp-odl-noha.local
------------
Succeeded: 2 (changed=2)
Failed:    0
------------
Total states run:     2
Total run time:  25.069 s
[NOTE] MaaS operations might take a long time, please be patient
tput: unknown terminal "unknown"
tput: unknown terminal "unknown"
mas01.mcp-odl-noha.local:
----------
          ID: maas_login_admin
    Function: cmd.run
        Name: maas-region apikey --username opnfv > /var/lib/maas/.maas_credentials
      Result: True
     Comment: Command "maas-region apikey --username opnfv > /var/lib/maas/.maas_credentials" run
     Started: 19:21:36.426407
    Duration: 5546.433 ms
     Changes:   
              ----------
              pid:
                  2204
              retcode:
                  0
              stderr:
              stdout:
----------
          ID: wait_for_machines_ready_or_deployed
    Function: module.run
        Name: maas.wait_for_machine_status
      Result: False
     Comment: Module function maas.wait_for_machine_status threw an exception. Exception: Machines:['cmp002', 'cmp003', 'cmp001', 'odl01', 'gtw01', 'ctl01']not in Ready|Deployed state
     Started: 19:21:41.976162
    Duration: 1508289.679 ms
     Changes:   

Summary for mas01.mcp-odl-noha.local
------------
Succeeded: 1 (changed=1)
Failed:    1
------------
Total states run:     2
Total run time: 1513.836 s
ERROR: Minions returned with non-zero exit code
[wait_for] ERROR: Failed after max attempts: ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i /var/lib/opnfv/mcp.rsa ubuntu@10.20.0.2 sudo             CI_DEBUG=false ERASE_ENV=0             /root/fuel/mcp/config/states/maas

[ERROR] MCP: Installation  of os-odl-nofeature-noha threw a fatal error!

(B
--------------------------------------------------------
Deployment is done!
Deployment failed!
Build step 'Execute shell' marked build as failure
New run name is '200 - Scenario: os-odl-nofeature-noha'
[PostBuildScript] - Executing post build scripts.
[PostBuildScript] - Build does not have any of the results [SUCCESS]. Did not execute build step #0.
[PostBuildScript] - Executing post build scripts.
[fuel-deploy-itri-pod1-daily-master] $ /bin/sh -xe /tmp/jenkins3548421890647824566.sh
+ echo export PROVISION_RESULT=FAIL
+ echo export INSTALLER=fuel
+ date +%Y-%m-%d %H:%M:%S.%3N
+ echo export TIMESTAMP_END='2019-09-19 03:47:02.193'
[fuel-deploy-itri-pod1-daily-master] $ /bin/bash /tmp/jenkins568831474902703229.sh
Installer: fuel provision result: FAIL
{ "installer": "fuel", "version": "master", "pod_name": "itri-pod1", "job_name": "fuel-deploy-itri-pod1-daily-master", "build_id": "200", "scenario": "os-odl-nofeature-noha", "upstream_job_name": "", "upstream_build_id":"", "criteria": "FAIL", "start_date": "2019-09-19 01:00:18.313", "stop_date": "2019-09-19 03:47:02.193", "details":"" }
++ gen_content
++ cat
+ curl -H 'Content-Type: application/json' -X POST -v -d '{
    "installer": "fuel",
    "version": "master",
    "pod_name": "itri-pod1",
    "job_name": "fuel-deploy-itri-pod1-daily-master",
    "build_id": "200",
    "scenario": "os-odl-nofeature-noha",
    "upstream_job_name": "",
    "upstream_build_id":"",
    "criteria": "FAIL",
    "start_date": "2019-09-19 01:00:18.313",
    "stop_date": "2019-09-19 03:47:02.193",
    "details":""
}' http://testresults.opnfv.org/test/api/v1/deployresults
Note: Unnecessary use of -X or --request, POST is already inferred.
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying 104.154.71.112...
* Connected to testresults.opnfv.org (104.154.71.112) port 80 (#0)
> POST /test/api/v1/deployresults HTTP/1.1
> Host: testresults.opnfv.org
> User-Agent: curl/7.47.0
> Accept: */*
> Content-Type: application/json
> Content-Length: 387
> 
} [387 bytes data]
* upload completely sent off: 387 out of 387 bytes

100   387    0     0  100   387      0   1356 --:--:-- --:--:-- --:--:--  1357< HTTP/1.1 200 OK
< Server: nginx/1.10.2
< Date: Wed, 18 Sep 2019 19:47:04 GMT
< Content-Type: application/json
< Content-Length: 79
< Connection: keep-alive
< 
{ [79 bytes data]

100   466  100    79  100   387    193    947 --:--:-- --:--:-- --:--:--   946
* Connection #0 to host testresults.opnfv.org left intact
{"href": "http://localhost:8082/api/v1/deployresults/5d8289b81b7bcc000e475626"}+ '[' FAIL == PASS ']'
+ exit 1
Email was triggered for: Always
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Request made to compress build log
Sending email to: Alexandru.Avadanii@enea.com
Sending email for trigger: Always
Request made to compress build log
Sending email to: Alexandru.Avadanii@enea.com
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Request made to compress build log
Sending email to: tbramwell@linuxfoundation.org agardner@linuxfoundation.org rgrigar@linuxfoundation.org
Finished: FAILURE