Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 2 additions & 44 deletions doc/source/operations/bifrost-hardware-inventory-management.rst
Original file line number Diff line number Diff line change
Expand Up @@ -105,51 +105,9 @@ To deprovision an existing hypervisor, run:
Removing a Hypervisor
---------------------

To remove a hypervisor without replacing it, proceed as follows. For compute
nodes, see :doc:`deprovision-compute` for a more detailed procedure.
To remove a hypervisor without replacing it, proceed as follows.

#. :ref:`Disable the hypervisor to avoid scheduling any new instances on it <taking-a-hypervisor-out-of-service>`
#. :ref:`Evacuate all instances <evacuating-all-instances>`
#. (optionally) Deprovision the hypervisor

.. code-block:: console

kayobe overcloud deprovision --limit <Hypervisor hostname>

.. warning::

Always use ``--limit`` with ``kayobe overcloud deprovision`` on a production
system. Running this command without a limit will deprovision all overcloud
hosts.

#. Physically remove the node from the deployment

#. Delete the node in Bifrost:

.. code-block:: console

docker exec -it bifrost_deploy bash
(bifrost-deploy)[root@seed bifrost-base]# export OS_CLOUD=bifrost
(bifrost-deploy)[root@seed bifrost-base]# openstack baremetal node delete <Hostname>

#. Delete the compute service in OpenStack:

.. code-block:: console

openstack compute service list | grep <Hypervisor hostname>
openstack compute service delete <Service ID>

#. Delete the network agents in OpenStack:

.. code-block:: console

openstack network agent list | grep <Hypervisor hostname>
openstack network agent delete <Agent IDs>

#. Remove the node from the Kayobe configuration. Ensure the node is removed
from the inventory and ``network-allocation.yml``. Other configuration files
may also be removed, but this is dependent on the deployment. Recursive
``grep`` can help here.
.. include:: deprovision-compute.rst
Comment thread
g0rgamesh marked this conversation as resolved.
Resolved

.. _evacuating-all-instances:

Expand Down
22 changes: 11 additions & 11 deletions doc/source/operations/deprovision-compute.rst
Original file line number Diff line number Diff line change
@@ -1,21 +1,21 @@
=============================
#############################
Deprovisioning a Compute Node
=============================
#############################

This procedure removes a compute node from service, drains it, removes all
traces of it from the control plane and Kayobe configuration, and finally
deprovisions the hardware in Bifrost.

Check the Hypervisor
====================
~~~~~~~~~~~~~~~~~~~~

.. code-block:: console

openstack compute service list --host <node>
openstack server list --all-projects --host <node>

Put It Into Maintenance Mode
============================
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. code-block:: console

Expand All @@ -29,7 +29,7 @@ Or use the OpenStack CLI:
openstack compute service set --disable --disable-reason "decommission <ticket>" <node> nova-compute

Drain the Node
==============
~~~~~~~~~~~~~~

.. code-block:: console

Expand All @@ -52,7 +52,7 @@ Unpause any paused instances.
openstack server migrate <id> --wait

Confirm It's Empty
==================
~~~~~~~~~~~~~~~~~~

On the to-be-decommissioned node:

Expand All @@ -61,15 +61,15 @@ On the to-be-decommissioned node:
docker exec nova_libvirt virsh list --all

Destroy Containers and Systemd Units
====================================
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. code-block:: console

sudo systemctl disable --now $(systemctl list-units 'kolla*' --all --plain --no-legend | awk '{print $1}')
docker stop $(docker ps -aq); docker rm $(docker ps -aq)

Remove It From the Control Plane
================================
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. warning::

Expand Down Expand Up @@ -103,7 +103,7 @@ On a controller, check if the chassis exists and delete it:
sudo docker exec ovn_sb_db ovn-sbctl --no-leader-only chassis-del <node>

Remove It From Kayobe and Monitoring
====================================
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Until this runs, Prometheus keeps scraping its exporters, and haproxy/fluentd
keep referencing it.
Expand All @@ -119,7 +119,7 @@ Remove it from ``etc/kayobe/inventory/overcloud`` and any group vars, then:
kayobe overcloud service reconfigure --kolla-limit controllers,monitoring

Deprovision the Hardware
========================
~~~~~~~~~~~~~~~~~~~~~~~~

From the seed.

Expand All @@ -132,7 +132,7 @@ From the seed.
sudo docker exec -it bifrost_deploy bash -c 'export OS_CLOUD=bifrost; openstack baremetal node delete <node>'

Verification
============
~~~~~~~~~~~~

.. code-block:: console

Expand Down
Loading