Nomad - Revision history

Leo: Added categories and navbox

2021-12-30T01:16:29Z

Added categories and navbox

← Older revision		Revision as of 01:16, 30 December 2021
Line 257:		Line 257:
	*Can tasks within the same group share a volume? Like local volumes with docker-compose (it just makes a docker volume that can be shared). This does not need to be persistent and I do not want to define <code>host_volumes</code> for something that's supposed to be temporary. Ephemeral_disk doesn't share a volume across tasks. Boo.		*Can tasks within the same group share a volume? Like local volumes with docker-compose (it just makes a docker volume that can be shared). This does not need to be persistent and I do not want to define <code>host_volumes</code> for something that's supposed to be temporary. Ephemeral_disk doesn't share a volume across tasks. Boo.
	*Can linked containers be made? Multiple tasks within a group? No. It looks like each container is separate and they can't look each other up. Use Consul's DNS server which can do service discovery. But how do you get this to work within docker?		*Can linked containers be made? Multiple tasks within a group? No. It looks like each container is separate and they can't look each other up. Use Consul's DNS server which can do service discovery. But how do you get this to work within docker?

			{{Navbox Linux}}
			[[Category:Linux]]
			[[Category:LinuxUtilities]]

Leo at 20:08, 17 February 2021

2021-02-17T20:08:18Z

← Older revision		Revision as of 20:08, 17 February 2021
Line 2:		Line 2:

	==Cheat sheet==		==Cheat sheet==
	~~Allocations are like pods, it contains one or more 'tasks'~~

	~~Tasks are just containers~~

	~~Job files contain all the information about the deployment.~~
	~~<br />~~
	{\| class="wikitable"		{\| class="wikitable"
	!Task		!Task
	!Command		!Command
	\|-		\|-
	\|Start an agent in dev mode		\|Start an agent
			<code>-dev</code> mode for testing
	\|<code>nomad agent -dev -bind 0.0.0.0 -log-level INFO</code>		\|<code>nomad agent -dev -bind 0.0.0.0 -log-level INFO</code>
	\|-		\|-
Line 92:		Line 87:
	\| lang = text		\| lang = text
	}}See: https://www.nomadproject.io/docs/drivers/docker#enabled-1		}}See: https://www.nomadproject.io/docs/drivers/docker#enabled-1

			By default with Docker tasks, network namespaces between tasks within a group are isolated. You can have multiple containers be on the same network namespace by specifying within <code>group -> network</code>, <code>mode = "bridge"</code>. This is useful if you were to run a web service application with a database backend but do not necessarily want to expose the database port.

	==Other==		==Other==

Leo at 06:51, 17 February 2021

2021-02-17T06:51:00Z

← Older revision		Revision as of 06:51, 17 February 2021
Line 1:		Line 1:
	~~HasiCorp~~ Nomad		HashiCorp Nomad

	==Cheat sheet==		==Cheat sheet==
Line 257:		Line 257:
	==Other Questions==		==Other Questions==

	*Can the docker driver run as specific users? As said in https://www.nomadproject.io/docs/job-specification/task#user, "Docker and rkt images specify their own default users." which is super crummy if true as you can't force containers to run as non-root.		*Can the docker driver run container as specific users? As said in https://www.nomadproject.io/docs/job-specification/task#user, "Docker and rkt images specify their own default users." which is super crummy if true as you can't force containers to run as non-root.
			*Can tasks within the same group share a volume? Like local volumes with docker-compose (it just makes a docker volume that can be shared). This does not need to be persistent and I do not want to define <code>host_volumes</code> for something that's supposed to be temporary. Ephemeral_disk doesn't share a volume across tasks. Boo.
			*Can linked containers be made? Multiple tasks within a group? No. It looks like each container is separate and they can't look each other up. Use Consul's DNS server which can do service discovery. But how do you get this to work within docker?

Leo at 06:44, 16 February 2021

2021-02-16T06:44:10Z

← Older revision		Revision as of 06:44, 16 February 2021
Line 12:		Line 12:
	!Command		!Command
	\|-		\|-
	\|Start an agent		\|Start an agent in dev mode
	\|<code>nomad agent -dev -bind 0.0.0.0 -log-level INFO</code>		\|<code>nomad agent -dev -bind 0.0.0.0 -log-level INFO</code>
	\|-		\|-
Line 56:		Line 56:
	# yum -y install nomad		# yum -y install nomad
	\| lang = terminal		\| lang = terminal
	}}		}}You can start Nomad immediately by running <code>nomad agent -dev-connect</code>. In dev mode, you can poke around with nomad with minimal risk since nothing you do will persist.

	===Configuration===		===Configuration===
Line 75:		Line 75:
	\| lang = text		\| lang = text
	}}		}}
	But when you start Nomad, you need to specify it with <code>-config</code>: <code>nomad agent ~~-dev-connect~~ -config=/etc/nomad.d~~/nomad.hcl~~</code>. You can also specify multiple locations with additional <code>-config</code> parameters. A config path can also be a directory, whereby Nomad will then load all <code>.hcl</code> and <code>.json</code> files within that directory.		But when you start Nomad, you need to specify it with <code>-config</code>: <code>nomad agent -config=/etc/nomad.d</code>. You can also specify multiple locations with additional <code>-config</code> parameters. A config path can also be a directory, whereby Nomad will then load all <code>.hcl</code> and <code>.json</code> files within that directory.

	Question: Does this file need to be replicated on all member nodes? Or does it replicate automatically somehow as diagramed at https://learn.hashicorp.com/tutorials/nomad/get-started-vocab?in=nomad/get-started#nomad-cluster		Question: Does this file need to be replicated on all member nodes? Or does it replicate automatically somehow as diagramed at https://learn.hashicorp.com/tutorials/nomad/get-started-vocab?in=nomad/get-started#nomad-cluster
Line 153:		Line 153:
	}}The downside/upside to this way of doing things is that the volume is tied to the Nomad agent (which is ideal if you want to only schedule this task on nodes that have the volume available), but adding additional <code>host_volumes</code> requires restarting the agent.		}}The downside/upside to this way of doing things is that the volume is tied to the Nomad agent (which is ideal if you want to only schedule this task on nodes that have the volume available), but adding additional <code>host_volumes</code> requires restarting the agent.

	===== Docker side note =====		=====Docker side note=====
	If you have enabled Docker volumes, you can also add additional volumes without needing to define <code>host_volumes</code> simply by listing the volumes within the config section of the task. For example, to mount <code>/local/volumes/container1</code> to <code>/data</code> in the container:		If you have enabled Docker volumes, you can also add additional volumes without needing to define <code>host_volumes</code> simply by listing the volumes within the config section of the task. For example, to mount <code>/local/volumes/container1</code> to <code>/data</code> in the container:
	{{Highlight		{{Highlight

Leo: /* Volumes */

2021-02-16T06:31:49Z

Volumes

← Older revision		Revision as of 06:31, 16 February 2021
Line 79:		Line 79:
	Question: Does this file need to be replicated on all member nodes? Or does it replicate automatically somehow as diagramed at https://learn.hashicorp.com/tutorials/nomad/get-started-vocab?in=nomad/get-started#nomad-cluster		Question: Does this file need to be replicated on all member nodes? Or does it replicate automatically somehow as diagramed at https://learn.hashicorp.com/tutorials/nomad/get-started-vocab?in=nomad/get-started#nomad-cluster

	==== Docker ====		====Docker====
	See: https://www.nomadproject.io/docs/drivers/docker#enabled-1		If you intend to use docker volumes, you need to enable it by specifying a custom docker plugin config. An example is given below. {{Highlight
			\| code = plugin "docker" {
	~~You may want to enable volumes~~		config {
			endpoint = "unix:///var/run/docker.sock"
			volumes {
			enabled = true
			selinuxlabel = "z"
			}
			}
			}
			\| lang = text
			}}See: https://www.nomadproject.io/docs/drivers/docker#enabled-1

	==Other==		==Other==
Line 93:		Line 102:
	===Persistent Volumes===		===Persistent Volumes===

	====~~Local~~ Volumes====		====Host Volumes====
	To have a container ~~use~~ a host volume:		To have a container mount a specific host volume, we need to:

	#Define a <code>host_volume</code> stanza in the Nomad agent client configuration. Restart the Nomad agent to apply.		#Define a <code>host_volume</code> stanza in the Nomad agent client configuration. Restart the Nomad agent to apply.
Line 100:		Line 109:
	#Define a <code>volume_mount</code> declaration in a task.		#Define a <code>volume_mount</code> declaration in a task.

	Step 1: Define a <code>host_volume</code> stanza within the client block in your Nomad configuration file. See: https://www.nomadproject.io/docs/configuration/client#host_volume-stanza		If you are using Docker, you may be interested in skipping all this and just mounting the volumes as you would with docker-compose in the sub section below.

			'''Step 1:''' Define a <code>host_volume</code> stanza within the client block in your Nomad configuration file. See: https://www.nomadproject.io/docs/configuration/client#host_volume-stanza
	{{Highlight		{{Highlight
	\| code = host_volume "mysql" {		\| code = host_volume "mysql" {
Line 110:		Line 121:
	Restart Nomad and then verify that the new host volume shows up when running <code>nomad node status -short -self</code>.		Restart Nomad and then verify that the new host volume shows up when running <code>nomad node status -short -self</code>.

	Step 2: Within the job/group, define the volume. Eg:		'''Step 2''': Within the job/group, define the volume. Eg:

	{{Highlight		{{Highlight
	\| code = volume "mysql" {		\| code = group "something" {
			volume "mysql" {
	type = "host"		type = "host"
	source = "mysql"		source = "mysql"
	read_only = false		read_only = false
	}		}
			...
			}
	\| lang = text		\| lang = text
	}}		}}

	Step 3: Within the task, ~~define~~:		'''Step 3''': Within the task, mount this volume:

	{{Highlight		{{Highlight
	\| code = volume_mount {		\| code = group "something" {
			...
			task "container" {
			...
			volume_mount {
	volume = "mysql"		volume = "mysql"
	destination = "/var/lib/mysql"		destination = "/var/lib/mysql"
	read_only = false		read_only = false
	}		}
			...
			}
			}
			\| lang = text
			}}The downside/upside to this way of doing things is that the volume is tied to the Nomad agent (which is ideal if you want to only schedule this task on nodes that have the volume available), but adding additional <code>host_volumes</code> requires restarting the agent.

			===== Docker side note =====
			If you have enabled Docker volumes, you can also add additional volumes without needing to define <code>host_volumes</code> simply by listing the volumes within the config section of the task. For example, to mount <code>/local/volumes/container1</code> to <code>/data</code> in the container:
			{{Highlight
			\| code = task "container" {
			driver = "docker"

			config {
			image = "..."

			volumes = [
			"/local/volumes/container1:/data"
			]
			...
			}
	\| lang = text		\| lang = text
	}}		}}

	==== NFS Volumes ====		====NFS Volumes====
	I am trying to replicate what I had with docker-compose where certain volumes were automatically mounted via NFS using the [https://github.com/ContainX/docker-volume-netshare ContainX/docker-volume-netshare] Docker plugin. Combing through the documentation and through some trial and error, I found out that you can define a specific volume driver to use with Docker.		I am trying to replicate what I had with docker-compose where certain volumes were automatically mounted via NFS using the [https://github.com/ContainX/docker-volume-netshare ContainX/docker-volume-netshare] Docker plugin. Combing through the documentation and through some trial and error, I found out that you can define a specific volume driver to use with Docker.

	In a task definition, specify a mount configuration as follows:		To get this working, install the <code>docker-volume-netshare</code> plugin and start the plugin as a service.

			In a task definition, specify a mount configuration as follows.
	{{Highlight		{{Highlight
	\| code = task "something" {		\| code = task "something" {
Line 166:		Line 206:
	\| lang = terminal		\| lang = terminal
	}}		}}
	The mount <code>source</code> value is analogous to the volume name you specify in the volumes definition in <code>docker-compose.yml</code> and is the directory name that netshare uses to mount the volume on the host machine. In the example above, when the container starts, it netshare will mount the volume to <code>/var/lib/docker-volumes/netshare/nfs/tv-volume</code>.		The <code>target</code> is the path of the volume in the container. The NFS server and export are defined as a the <code>share</code> in the driver option. The mount <code>source</code> value is analogous to the volume name you specify in the volumes definition in <code>docker-compose.yml</code> and is the directory name that netshare uses to mount the volume on the host machine. In the example above, when the container starts, it netshare will mount the volume to <code>/var/lib/docker-volumes/netshare/nfs/tv-volume</code>.

	If you get a "<code>volumes are not enabled</code>" error, you need to enable docker volumes in the docker plugins configuration. Edit <code>nomad.hcl</code> and add:		If you get a <code>volumes are not enabled; cannot mount host paths:</code> error, you need to enable docker volumes in the docker plugins configuration. Edit <code>nomad.hcl</code> and add:
	{{Highlight		{{Highlight
	\| code = plugin "docker" {		\| code = plugin "docker" {

Leo at 06:05, 16 February 2021

2021-02-16T06:05:57Z

@@ Line 78: / Line 78: @@
 Question: Does this file need to be replicated on all member nodes? Or does it replicate automatically somehow as diagramed at https://learn.hashicorp.com/tutorials/nomad/get-started-vocab?in=nomad/get-started#nomad-cluster
 ==Other==
@@ Line 91: / Line 96: @@
 To have a container use a host volume:
-# Define a <code>host_volume</code> stanza in the Nomad agent client configuration. Restart the Nomad agent to apply.
+#Define a <code>host_volume</code> stanza in the Nomad agent client configuration. Restart the Nomad agent to apply.
-# Define a <code>volume</code> declaration within a task's group.
+#Define a <code>volume</code> declaration within a task's group.
-# Define a <code>volume_mount</code> declaration in a task.
+#Define a <code>volume_mount</code> declaration in a task.
 Step 1: Define a <code>host_volume</code> stanza within the client block in your Nomad configuration file. See: https://www.nomadproject.io/docs/configuration/client#host_volume-stanza
@@ Line 127: / Line 132: @@
 }}
-====Open Questions====
+==== NFS Volumes ====
-*How do I get a container with a persistent volume via NFS?
+If you get a "<code>volumes are not enabled</code>" error, you need to enable docker volumes in the docker plugins configuration. Edit <code>nomad.hcl</code> and add:
 ===Traefik load balancing===
@@ Line 164: / Line 215: @@
 *Can we have the load balancing Traefik instance be an external service? If so, how?
-== Other Questions ==
+==Other Questions==
-* Can the docker driver run as specific users? As said in https://www.nomadproject.io/docs/job-specification/task#user, "Docker and rkt images specify their own default users." which is super crummy if true as you can't force containers to run as non-root.
+*Can the docker driver run as specific users? As said in https://www.nomadproject.io/docs/job-specification/task#user, "Docker and rkt images specify their own default users." which is super crummy if true as you can't force containers to run as non-root.

Leo: /* Persistent Volumes */

2021-02-16T04:39:11Z

Persistent Volumes

← Older revision		Revision as of 04:39, 16 February 2021
Line 37:		Line 37:
	\|}		\|}

	== Concepts ==		==Concepts==
	See: https://learn.hashicorp.com/tutorials/nomad/get-started-vocab?in=nomad/get-started		See: https://learn.hashicorp.com/tutorials/nomad/get-started-vocab?in=nomad/get-started

	* '''Task''': Smallest unit of work and is executed by a task driver. Eg. a container running via Docker		*'''Task''': Smallest unit of work and is executed by a task driver. Eg. a container running via Docker
	* '''Group''': A set of tasks that must run together. Eg. a web server and a database server		*'''Group''': A set of tasks that must run together. Eg. a web server and a database server
	* '''Job''': A set of (one or more) groups. Used to constrain the scheduler, set update strategies, ACL		*'''Job''': A set of (one or more) groups. Used to constrain the scheduler, set update strategies, ACL

	When defining a job, you:		When defining a job, you:

	# Specify a job definition		#Specify a job definition
	# Within the job, you specify one or more groups		#Within the job, you specify one or more groups
	# Within each group, you specify its replica (count=#), task and service		#Within each group, you specify its replica (count=#), task and service

	== Installation ==		==Installation==
	Installation instructions are outlined at https://www.nomadproject.io/docs/install. For CentOS, run:		Installation instructions are outlined at https://www.nomadproject.io/docs/install. For CentOS, run:
	{{Highlight		{{Highlight
Line 58:		Line 58:
	}}		}}

	=== Configuration ===		===Configuration===
	The Nomad configuration file is located at: <code>/etc/nomad.d/nomad.hcl</code>.		The Nomad configuration file is located at: <code>/etc/nomad.d/nomad.hcl</code>.
	{{Highlight		{{Highlight
Line 77:		Line 77:
	But when you start Nomad, you need to specify it with <code>-config</code>: <code>nomad agent -dev-connect -config=/etc/nomad.d/nomad.hcl</code>. You can also specify multiple locations with additional <code>-config</code> parameters. A config path can also be a directory, whereby Nomad will then load all <code>.hcl</code> and <code>.json</code> files within that directory.		But when you start Nomad, you need to specify it with <code>-config</code>: <code>nomad agent -dev-connect -config=/etc/nomad.d/nomad.hcl</code>. You can also specify multiple locations with additional <code>-config</code> parameters. A config path can also be a directory, whereby Nomad will then load all <code>.hcl</code> and <code>.json</code> files within that directory.

	Question: Does this file need to be replicated on all member nodes?		Question: Does this file need to be replicated on all member nodes? Or does it replicate automatically somehow as diagramed at https://learn.hashicorp.com/tutorials/nomad/get-started-vocab?in=nomad/get-started#nomad-cluster

	==Other==		==Other==
Line 88:		Line 88:
	===Persistent Volumes===		===Persistent Volumes===

	==== Local Volumes ====		====Local Volumes====
	~~Volumes local to the~~ host ~~running the task are defined as <code>host_volumes</code>. These are defined within the Nomad configuration within the <code>client</code> stanza/block and requires the Nomad server to be restarted.~~		To have a container use a host volume:

			# Define a <code>host_volume</code> stanza in the Nomad agent client configuration. Restart the Nomad agent to apply.
			# Define a <code>volume</code> declaration within a task's group.
			# Define a <code>volume_mount</code> declaration in a task.

	Define a <code>host_volume</code> stanza within the client block (https://www.nomadproject.io/docs/configuration/client#host_volume-stanza~~) in your Nomad configuration file.~~		Step 1: Define a <code>host_volume</code> stanza within the client block in your Nomad configuration file. See: https://www.nomadproject.io/docs/configuration/client#host_volume-stanza
	{{Highlight		{{Highlight
	\| code = host_volume "mysql" {		\| code = host_volume "mysql" {
Line 102:		Line 105:
	Restart Nomad and then verify that the new host volume shows up when running <code>nomad node status -short -self</code>.		Restart Nomad and then verify that the new host volume shows up when running <code>nomad node status -short -self</code>.

	==== Open Questions ====		Step 2: Within the job/group, define the volume. Eg:

			{{Highlight
			\| code = volume "mysql" {
			type = "host"
			source = "mysql"
			read_only = false
			}
			\| lang = text
			}}

			Step 3: Within the task, define:

			{{Highlight
			\| code = volume_mount {
			volume = "mysql"
			destination = "/var/lib/mysql"
			read_only = false
			}
			\| lang = text
			}}

			====Open Questions====

	*How do I get a container with a persistent volume via NFS?		*How do I get a container with a persistent volume via NFS?
Line 138:		Line 163:

	*Can we have the load balancing Traefik instance be an external service? If so, how?		*Can we have the load balancing Traefik instance be an external service? If so, how?

			== Other Questions ==

			* Can the docker driver run as specific users? As said in https://www.nomadproject.io/docs/job-specification/task#user, "Docker and rkt images specify their own default users." which is super crummy if true as you can't force containers to run as non-root.

Leo at 00:58, 16 February 2021

2021-02-16T00:58:24Z

@@ Line 35: / Line 35: @@
 |Examine an allocation's logs
 |<code>nomad alloc logs allocation-id task-name</code>
-|}<br />
+|}
 ==Other==
@@ Line 43: / Line 86: @@
 Nomad appears to just detect this automatically, like magic.
-=== NFS Persistent Volumes ===
+===Persistent Volumes===
-* How do I get a container with a persistent volume via NFS?
+===Traefik load balancing===
 Configure Traefik with the <code>consulCatalog</code> provider in the <code>traefik.toml</code> configuration file:
 {{Highlight
@@ Line 74: / Line 135: @@
 Traefik will then be notified of the new backends when changes to this service happens via Consul.
-==== Open Questions ====
+====Open Questions====
-* Can we have the load balancing Traefik instance be an external service? If so, how?
+*Can we have the load balancing Traefik instance be an external service? If so, how?

Leo: /* Cheat sheet */

2021-02-16T00:26:15Z

Cheat sheet

@@ Line 1: / Line 1: @@
 HasiCorp Nomad
-== Cheat sheet ==
+==Cheat sheet==
 Allocations are like pods, it contains one or more 'tasks'
@@ Line 9: / Line 9: @@
 <br />
 {| class="wikitable"
+!Task
+!Command
 |-
 |Start an agent
@@ Line 35: / Line 35: @@
 |Examine an allocation's logs
 |<code>nomad alloc logs allocation-id task-name</code>
-|}
+|}<br />
-== Other ==
+===Consul integration===
 Start consul: <code>consul agent -dev</code>
 Nomad appears to just detect this automatically, like magic.

Leo: Initial Content

2021-02-16T00:20:36Z

Initial Content

New page

HasiCorp Nomad

== Cheat sheet ==
Allocations are like pods, it contains one or more 'tasks'

Tasks are just containers

Job files contain all the information about the deployment.
<br />
{| class="wikitable"
!
!
|-
|Start an agent
|<code>nomad agent -dev -bind 0.0.0.0 -log-level INFO</code>
|-
|Show node status
|<code>nomad node status</code>
|-
|Show connected members
|<code>nomad server members</code>
|-
|Create an example job
|<code>nomad job init</code>
|-
|Run a job
|<code>nomad job run example.nomad</code>
|-
|Show a job's status
|<code>nomad job status example</code>
|-
|Examine an allocation
|<code>nomad alloc status allocation-id</code>
|-
|Examine an allocation's logs
|<code>nomad alloc logs allocation-id task-name</code>
|}

== Other ==

=== Consul integration ===
Start consul: <code>consul agent -dev</code>

Nomad appears to just detect this automatically, like magic.