Compare commits

..

33 Commits

Author SHA1 Message Date
mergify[bot]
232cf793d9
Add/update DCO, CoC, contributing ... files (#93)
## Description

Update non code files to be uniformish amongst all repos.

## Why is this needed

Ensures our repos are all licensed correctly and contain similar/minimum
contribution info files.
2021-08-05 18:42:05 +00:00
Manuel Mendez
f3bc190943 Add in repo DCO file
Signed-off-by: Manuel Mendez <mmendez@equinix.com>
2021-07-19 10:57:45 -04:00
Manuel Mendez
232544f863 Add CONTRIBUTING.md
Signed-off-by: Manuel Mendez <mmendez@equinix.com>
2021-07-19 10:57:45 -04:00
Jacob Weinstock
1ebcf482de
Merge pull request #88 from micahhausler/baremetal
Enable skipping of network in setup.sh
2021-06-25 11:03:25 -06:00
Micah Hausler
7182ca0811 Enable skipping of network in setup.sh
* Fixed path to deploy dir in log message
* Allow TINKERBELL_CIDR and TINKERBELL_HOST_IP to be overridable
* Set environment variables for tink cli in .env

Signed-off-by: Micah Hausler <mhausler@amazon.com>
2021-06-25 12:57:14 -04:00
mergify[bot]
1760df0caf
deploy: Bump libvirt vagrant box version to 0.2.0 (#83)
## Description

Upgrade libvirt box version used in vagrant setup

## Why is this needed

Vagrant will continue to use the buggy v0.1.0 vagrant box until this is merged.
2021-05-04 09:14:24 +00:00
Manuel Mendez
9bea6a01df deploy: Bump libvirt vagrant box version to 0.2.0
https://app.vagrantup.com/tinkerbelloss/boxes/sandbox-ubuntu1804/versions/0.2.0
exist now as a fix for #59 and #62.

Signed-off-by: Manuel Mendez <mmendez@equinix.com>
2021-05-03 18:03:27 +00:00
mergify[bot]
4add7eef56
Fix empty docker-compose in basebox (#81)
## Description

Ensures docker-compose is correctly downloaded.
Also adds some better debuggability to setup.sh and the vagrant provision script.
A bunch of misc clean ups following the boy scout rule (leave things better than you found them)

## Why is this needed

Fixes: #59 

## How Has This Been Tested?

`vagrant up provisioner` now works

## How are existing users impacted? What migration steps/scripts do we need?

Fixes a bug where the vagrant sandbox wasn't working.

## Checklist:

I have:

- [ ] updated the documentation and/or roadmap (if required)
- [ ] added unit or e2e tests
- [ ] provided instructions on how to upgrade
2021-04-29 17:19:59 +00:00
Manuel Mendez
7e2296df94 setup: Correct misspelling fist -> first
Signed-off-by: Manuel Mendez <mmendez@equinix.com>
2021-04-29 15:16:43 +00:00
Manuel Mendez
ffbb92909b setup: Add blank file check to check_command
This way we can better gaurd against empty files as seen
in the previous commits message.

Signed-off-by: Manuel Mendez <mmendez@equinix.com>
2021-04-28 20:14:29 +00:00
Manuel Mendez
549e540671 vagrant: Fix basebox having corrupt docker-compose binary
This fixes the vagrant based sandbox from not working. This was particularly
annoying to track down because of not having `set -x` in `setup.sh` but
what looks like xtrace output in stderr. The xtrace output on stderr
was actually from the `generate_certificates` container:

```
    provisioner: 2021/04/26 21:22:32 [INFO] signed certificate with serial number 142120228981443865252746731124927082232998754394
    provisioner: + cat
    provisioner:  server.pem
    provisioner:  ca.pem
    provisioner: + cmp
    provisioner:  -s
    provisioner:  bundle.pem.tmp
    provisioner:  bundle.pem
    provisioner: + mv
    provisioner:  bundle.pem.tmp
    provisioner:  bundle.pem
    provisioner: Error: No such object:
==> provisioner: Clearing any previously set forwarded ports...
==> provisioner: Removing domain...
The SSH command responded with a non-zero exit status. Vagrant
assumes that this means the command failed. The output for this command
should be in the log above. Please read the output to determine what
went wrong.
```
I ended up doubting the `if ! cmp` blocks until I added `set -euxo pipefail` and
the issue was pretty obviously in docker-compose land.

```
$ vagrant destroy -f; vagrant up provisioner
==> worker: Domain is not created. Please run `vagrant up` first.
==> provisioner: Domain is not created. Please run `vagrant up` first.
Bringing machine 'provisioner' up with 'libvirt' provider...
==> provisioner: Checking if box 'tinkerbelloss/sandbox-ubuntu1804' version '0.1.0' is up to date...
==> provisioner: Creating image (snapshot of base box volume).
==> provisioner: Creating domain with the following settings...
...
    provisioner: 2021/04/27 18:20:13 [INFO] signed certificate with serial number 138080403356863347716407921665793913032297783787
    provisioner: + cat server.pem ca.pem
    provisioner: + cmp -s bundle.pem.tmp bundle.pem
    provisioner: + mv bundle.pem.tmp bundle.pem
    provisioner: + local certs_dir=/etc/docker/certs.d/192.168.1.1
    provisioner: + cmp --quiet /vagrant/deploy/state/certs/ca.pem /vagrant/deploy/state/webroot/workflow/ca.pem
    provisioner: + cp /vagrant/deploy/state/certs/ca.pem /vagrant/deploy/state/webroot/workflow/ca.pem
    provisioner: + cmp --quiet /vagrant/deploy/state/certs/ca.pem /etc/docker/certs.d/192.168.1.1/tinkerbell.crt
    provisioner: + [[ -d /etc/docker/certs.d/192.168.1.1/ ]]
    provisioner: + cp /vagrant/deploy/state/certs/ca.pem /etc/docker/certs.d/192.168.1.1/tinkerbell.crt
    provisioner: + setup_docker_registry
    provisioner: + local registry_images=/vagrant/deploy/state/registry
    provisioner: + [[ -d /vagrant/deploy/state/registry ]]
    provisioner: + mkdir -p /vagrant/deploy/state/registry
    provisioner: + start_registry
    provisioner: + docker-compose -f /vagrant/deploy/docker-compose.yml up --build -d registry
    provisioner: + check_container_status registry
    provisioner: + local container_name=registry
    provisioner: + local container_id
    provisioner: ++ docker-compose -f /vagrant/deploy/docker-compose.yml ps -q registry
    provisioner: + container_id=
    provisioner: + local start_moment
    provisioner: + local current_status
    provisioner: ++ docker inspect '' --format '{{ .State.StartedAt }}'
    provisioner: Error: No such object:
    provisioner: + start_moment=
    provisioner: + finish
    provisioner: + rm -rf /tmp/tmp.ve3XJ7qtgA
```

Notice that `container_id` is empty. This turns out to be because
`docker-compose` is an empty file!

```
vagrant@provisioner:/vagrant/deploy$ docker-compose up --build registry
vagrant@provisioner:/vagrant/deploy$ which docker-compose
/usr/local/bin/docker-compose
vagrant@provisioner:/vagrant/deploy$ docker-compose -h
vagrant@provisioner:/vagrant/deploy$ file /usr/local/bin/docker-compose
/usr/local/bin/docker-compose: empty
```

So with the following test patch:

```diff
diff --git a/deploy/vagrant/scripts/tinkerbell.sh b/deploy/vagrant/scripts/tinkerbell.sh
index 915f27f..dcb379c 100644
--- a/deploy/vagrant/scripts/tinkerbell.sh
+++ b/deploy/vagrant/scripts/tinkerbell.sh
@@ -34,6 +34,14 @@ setup_nat() (
 main() (
 	export DEBIAN_FRONTEND=noninteractive

+	local name=docker-compose-$(uname -s)-$(uname -m)
+	local url=https://github.com/docker/compose/releases/download/1.26.0/$name
+	curl -fsSLO "$url"
+	curl -fsSLO "$url.sha256"
+	sha256sum -c <"$name.sha256"
+	chmod +x "$name"
+	sudo mv "$name" /usr/local/bin/docker-compose
+
 	if ! [[ -f ./.env ]]; then
 		./generate-env.sh eth1 >.env
 	fi
```

We can try again and we're back to a working state:

```
$ vagrant destroy -f; vagrant up provisioner
==> worker: Domain is not created. Please run `vagrant up` first.
==> provisioner: Domain is not created. Please run `vagrant up` first.
Bringing machine 'provisioner' up with 'libvirt' provider...
==> provisioner: Checking if box 'tinkerbelloss/sandbox-ubuntu1804' version '0.1.0' is up to date...
==> provisioner: Creating image (snapshot of base box volume).
==> provisioner: Creating domain with the following settings...
...
    provisioner: + setup_docker_registry
    provisioner: + local registry_images=/vagrant/deploy/state/registry
    provisioner: + [[ -d /vagrant/deploy/state/registry ]]
    provisioner: + mkdir -p /vagrant/deploy/state/registry
    provisioner: + start_registry
    provisioner: + docker-compose -f /vagrant/deploy/docker-compose.yml up --build -d registry
    provisioner: Creating network "deploy_default" with the default driver
    provisioner: Creating volume "deploy_postgres_data" with default driver
    provisioner: Building registry
    provisioner: Step 1/7 : FROM registry:2.7.1
...
    provisioner: Successfully tagged deploy_registry:latest
    provisioner: Creating deploy_registry_1 ...
Creating deploy_registry_1 ... done
    provisioner: + check_container_status registry
    provisioner: + local container_name=registry
    provisioner: + local container_id
    provisioner: ++ docker-compose -f /vagrant/deploy/docker-compose.yml ps -q registry
    provisioner: + container_id=2e3d9557fd4c0d7f7e1c091b957a0033d23ebb93f6c8e5cdfeb8947b2812845c
...
    provisioner: + sudo -iu vagrant docker login --username=admin --password-stdin 192.168.1.1
    provisioner: WARNING! Your password will be stored unencrypted in /home/vagrant/.docker/config.json.
    provisioner: Configure a credential helper to remove this warning. See
    provisioner: https://docs.docker.com/engine/reference/commandline/login/#credentials-store
    provisioner: Login Succeeded
    provisioner: + set +x
    provisioner: NEXT:  1. Enter /vagrant/deploy and run: source ../.env; docker-compose up -d
    provisioner:        2. Try executing your fist workflow.
    provisioner:           Follow the steps described in https://tinkerbell.org/examples/hello-world/ to say 'Hello World!' with a workflow.
```

:toot:

Except that my results are not due to the way docker-compose is being installed
at all. After still running into this issue when using a box built with the new
install method I was still seeing empty docker-compose files. I ran a bunch of
experiments to try and figure out what is going on. The issue is strictly
in vagrant-libvirt since vagrant-virtualbox works fine. Turns out data isn't
being flushed back to disk at shutdown. Both calling `sync` or writing multiple
copies of the binary to the fs (3x at least) ended up working. Then I was informed
of a known vagrant-libvirt issue which matches this behavior, https://github.com/vagrant-libvirt/vagrant-libvirt/issues/1013!

Fixes #59

Signed-off-by: Manuel Mendez <mmendez@equinix.com>
2021-04-28 19:54:35 +00:00
Manuel Mendez
4a59c96463 vagrant: Ensure the whats_next message is printed at the end
The tinkerbell.sh script ends up doing some other work after
calling setup.sh and has set -x enabled so the whats_next message
is likely to be missed. So now save it for later reading as the last
thing done.

Signed-off-by: Manuel Mendez <mmendez@equinix.com>
2021-04-27 20:05:49 +00:00
Manuel Mendez
51777df36c setup: Add xtrace and pipefail to set options
pipefail for more safety and xtrace for better debuggability.
The missing xtrace here is likely what led to the docker-compose
issue going unfixed for so long as the last bit of output was
from the gencerts container and did not make any sense (because it
wasn't the issue :D ).

Signed-off-by: Manuel Mendez <mmendez@equinix.com>
2021-04-27 20:05:49 +00:00
Manuel Mendez
5eceec91ed box: make lists be multiline and with same line ending
Better for adding/removing things this way.

Signed-off-by: Manuel Mendez <mmendez@equinix.com>
2021-04-27 20:05:49 +00:00
Manuel Mendez
8e5430bfd1 generate-env: Use <<-EOF to indent the heredoc
Indentation is helpful to know a function's scope. Not indenting the
heredoc makes scanning harder.

Signed-off-by: Manuel Mendez <mmendez@equinix.com>
2021-04-27 20:05:48 +00:00
Manuel Mendez
0fff3e6d7f sh: Make use of bashisms in bash scripts
Both [[ ]] and (( )) bashisms are better than the alternative
in POSIX sh, since they are builtin and don't suffer from quoting
or number-of-args issues.

Signed-off-by: Manuel Mendez <mmendez@equinix.com>
2021-04-27 20:04:38 +00:00
Manuel Mendez
b8d94f5278 setup.sh: Quote full args instead of just bash variables
More in line with the rest of scripts and is easier to mentally parse.

Signed-off-by: Manuel Mendez <mmendez@equinix.com>
2021-04-27 20:04:38 +00:00
Manuel Mendez
88bf5771ea vagrant: Use source instead of . for better grepability.
Signed-off-by: Manuel Mendez <mmendez@equinix.com>
2021-04-27 20:04:38 +00:00
mergify[bot]
28a236376f
Rename "generate-envrc" to "generate-env" (#79)
Rename "generate-envrc" to "generate-env"
2021-04-20 08:59:26 +00:00
Gaurav Gahlot
c40086d221
rename generate-envrc to generate-env
Signed-off-by: Gaurav Gahlot <gauravgahlot0107@gmail.com>
2021-04-16 22:21:10 +05:30
Gianluca Arbezzano
712f3eb38a
Delete CODEOWNERS
This was an attempt to set ownership. It didn't work
2021-04-16 13:17:39 +02:00
mergify[bot]
661855eb26
Fix setup.sh to work when .nat_interface doesn't exist (#78)
## Description

This is a follow-up to #76 which introduced a failure:
```
provisioner: ./setup.sh: line 117: NAT_INTERFACE: unbound variable
```

## Why is this needed

Unbreak `setup.sh` when used by Vagrant

Fixes #77 

## How Has This Been Tested?

I used the following simple test case. It works now that the variable is declared first, but still breaks as reported without the fix.
```bash
#!/bin/bash
set -eu
NAT_INTERFACE=""
if [ -r .nat_interface ]; then
	NAT_INTERFACE=$(cat .nat_interface)
fi
if [ -n "$NAT_INTERFACE" ] && ip addr show "$NAT_INTERFACE" &>/dev/null; then
	echo "$NAT_INTERFACE"
fi
```

## How are existing users impacted? What migration steps/scripts do we need?

Vagrant users are currently broken as reported in the community Slack.

## Checklist:

I have:

- [ ] updated the documentation and/or roadmap (if required)
- [ ] added unit or e2e tests
- [ ] provided instructions on how to upgrade
2021-04-12 17:49:20 +00:00
Nahum Shalman
4243501dca Fix setup.sh to work when .nat_interface doesn't exist
Signed-off-by: Nahum Shalman <nshalman@equinix.com>
2021-04-12 16:35:08 +00:00
mergify[bot]
3fc23c58eb
Terraform in Equinix Metal: Fix NAT to reference correct interfaces (#76)
## Description

The NAT setup commands assume that the interface is named eth1, when clearly from the [documentation](https://github.com/tinkerbell/tinkerbell-docs/blame/master/docs/setup/equinix-metal-terraform.md#L118) it is named `enp1s0f1`. This commit fixes the NAT setup commands accordingly.

## Why is this needed

NAT doesn't work by default on Equinix Metal when following the documentation

## How Has This Been Tested?

- [x] Tested with Terraform in Equinix Metal

## How are existing users impacted? What migration steps/scripts do we need?

Existing sandboxes (that are broken) should either be rebuilt, or can run the commands manually to enable NAT

## Checklist:

I have:

- [ ] updated the documentation and/or roadmap (if required)
- [ ] added unit or e2e tests
- [ ] provided instructions on how to upgrade
2021-04-08 20:24:22 +00:00
Nahum Shalman
4d13239d77 Fix NAT to reference correct interfaces
This moves the NAT commands from terraform to setup.sh

Signed-off-by: Nahum Shalman <nshalman@equinix.com>
2021-04-08 16:17:38 +00:00
mergify[bot]
5347fe6da7
Add jq to the nix-shell environment (#75)
Signed-off-by: Nahum Shalman <nshalman@equinix.com>

## Description

Add `jq` to the nix-shell environment

## Why is this needed

There are bits of documentation that use the sandbox and reference using `jq` from the command line.
This makes them work nicely.

## How Has This Been Tested?
On NixOS running `nix-shell` now has `jq` in the PATH.

## How are existing users impacted? What migration steps/scripts do we need?

N/A

## Checklist:

I have:

- [ ] updated the documentation and/or roadmap (if required)
- [ ] added unit or e2e tests
- [ ] provided instructions on how to upgrade
2021-04-06 18:14:15 +00:00
Nahum Shalman
89e49554be Add jq to the nix-shell environment
Signed-off-by: Nahum Shalman <nshalman@equinix.com>
2021-04-06 14:34:14 +00:00
mergify[bot]
cc5e0be668
Update Terraform to 0.14 in shell.nix (#72)
## Description

This change causes Nix to pull in Terraform v0.14 rather than v0.12 when users run `nix-shell`

## Why is this needed

Without this change I get this error on both Mac and NixOS:
```
[nix-shell:~/sandbox/deploy/terraform]$ terraform init --upgrade

Warning: Provider source not supported in Terraform v0.12

  on main.tf line 4, in terraform:
   4:     metal = {
   5:       source  = "equinix/metal"
   6:       version = "1.0.0"
   7:     }

A source was declared for provider metal. Terraform v0.12 does not support the
provider source attribute. It will be ignored.

(and 2 more similar warnings elsewhere)


Error: Unsupported Terraform Core version

This configuration does not support Terraform version 0.12.30. To proceed,
either choose another supported Terraform version or update the root module's
version constraint. Version constraints are normally set for good reason, so
updating the constraint may lead to other errors or unexpected behavior.
```

## How Has This Been Tested?

I used this on both a Mac and a NixOS machine using `nix-shell`

## How are existing users impacted? What migration steps/scripts do we need?

Running `nix-shell` on existing checkouts will now pull down a newer version of Terraform.

## Checklist:

I have:

- [ ] updated the documentation and/or roadmap (if required)
- [ ] added unit or e2e tests
- [ ] provided instructions on how to upgrade
2021-03-24 15:06:57 +00:00
Nahum Shalman
edd0d06b93 Update Terraform to 0.14 in shell.nix
Signed-off-by: Nahum Shalman <nshalman@equinix.com>
2021-03-24 14:21:39 +00:00
Gianluca Arbezzano
cfb1781a69 chore: install btrfs-progs as an attempt to make get-binaries script to work
Signed-off-by: Gianluca Arbezzano <gianarb92@gmail.com>
2021-03-24 13:39:36 +01:00
Gianluca Arbezzano
7d878a8e8e
Merge pull request #70 from tinkerbell/install-btrfs-devel
Install btrfs devel
2021-03-24 13:31:00 +01:00
Gianluca Arbezzano
b52a90b666 Install btrfs devel
https://github.com/containerd/containerd/issues/3488

As explained here it looks like a requirement now for Ubuntu

Signed-off-by: Gianluca Arbezzano <gianarb92@gmail.com>
2021-03-24 13:25:12 +01:00
mergify[bot]
4a76489a3e
Revert osie because the latest version broke tink-worker installation (#69)
As I explained here
https://github.com/tinkerbell/sandbox/pull/66#issuecomment-803009169 the
current OSIE on master broke how tink-worker gets installed in sandbox.

For a series of bad habits, the PR got merged even if e2e tests are
broken leaving sandbox/master to a not working state

This commit reverts OSIE back to a fully operational version
2021-03-24 12:17:01 +00:00
15 changed files with 298 additions and 165 deletions

View File

@ -7,7 +7,7 @@ jobs:
release: release:
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- run: sudo apt-get update && sudo apt install -y libgpgme-dev libassuan-dev libdevmapper-dev - run: sudo apt-get update && sudo apt install -y libgpgme-dev libassuan-dev libdevmapper-dev btrfs-progs libbtrfs-dev
- name: Checkout code - name: Checkout code
uses: actions/checkout@v2 uses: actions/checkout@v2
- uses: actions/setup-go@v2 - uses: actions/setup-go@v2

View File

@ -1,6 +0,0 @@
# These owners will be the default owners for everything in
# the repo. Unless a later match takes precedence,
# @global-owner1 and @global-owner2 will be requested for
# review when someone opens a pull request.
* @gauravgahlot @gianarb

45
CONTRIBUTING.md Normal file
View File

@ -0,0 +1,45 @@
## Hello Contributors!
Thanks for your interest!
We're so glad you're here.
### Important Resources
#### bugs: [https://github.com/tinkerbell/sandbox/issues](https://github.com/tinkerbell/sandbox/issues)
### Code of Conduct
Please read and understand the code of conduct found [here](https://github.com/tinkerbell/.github/blob/master/CODE_OF_CONDUCT.md).
### DCO Sign Off
Please read and understand the DCO found [here](docs/DCO.md).
### Environment Details
Building is handled by `make`, please see the [Makefile](Makefile) for available targets.
#### Nix
This repo's build environment can be reproduced using `nix`.
##### Install Nix
Follow the [Nix installation](https://nixos.org/download.html) guide to setup Nix on your box.
##### Load Dependencies
Loading build dependencies is as simple as running `nix-shell` or using [lorri](https://github.com/nix-community/lorri).
If you have `direnv` installed the included `.envrc` will make that step automatic.
### How to Submit Change Requests
Please submit change requests and / or features via [Issues](https://github.com/tinkerbell/sandbox/issues).
There's no guarantee it'll be changed, but you never know until you try.
We'll try to add comments as soon as possible, though.
### How to Report a Bug
Bugs are problems in code, in the functionality of an application or in its UI design; you can submit them through [Issues](https://github.com/tinkerbell/sandbox/issues).
## Code Style Guides

View File

@ -59,7 +59,7 @@ const headerFile = `#!/bin/bash
# This file is generated by an utility called bump-version in # This file is generated by an utility called bump-version in
# tinkerbell/sandbox. # tinkerbell/sandbox.
# This file gets used from generate-envrc.sh but it is also used standalone by # This file gets used from generate-env.sh but it is also used standalone by
# automation that wants to get the version of the programs currently supported # automation that wants to get the version of the programs currently supported
# in sandbox # in sandbox

View File

@ -2,7 +2,7 @@
# This file is generated by an utility called bump-version in # This file is generated by an utility called bump-version in
# tinkerbell/sandbox. # tinkerbell/sandbox.
# This file gets used from generate-envrc.sh but it is also used standalone by # This file gets used from generate-env.sh but it is also used standalone by
# automation that wants to get the version of the programs currently supported # automation that wants to get the version of the programs currently supported
# in sandbox # in sandbox

View File

@ -57,8 +57,8 @@ resource "null_resource" "tink_directory" {
} }
provisioner "file" { provisioner "file" {
source = "../../generate-envrc.sh" source = "../../generate-env.sh"
destination = "/root/tink/generate-envrc.sh" destination = "/root/tink/generate-env.sh"
} }
provisioner "file" { provisioner "file" {
@ -71,12 +71,9 @@ resource "null_resource" "tink_directory" {
destination = "/root/tink" destination = "/root/tink"
} }
provisioner "remote-exec" { provisioner "file" {
inline = [ source = "nat_interface"
"iptables -A FORWARD -i eth1 -o bond0 -j ACCEPT", destination = "/root/tink/.nat_interface"
"iptables -A FORWARD -i bond0 -o eth1 -m state --state ESTABLISHED,RELATED -j ACCEPT",
"iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE",
]
} }
provisioner "remote-exec" { provisioner "remote-exec" {

View File

@ -0,0 +1 @@
bond0

View File

@ -26,7 +26,7 @@ Vagrant.configure('2') do |config|
config.vm.define :provisioner do |provisioner| config.vm.define :provisioner do |provisioner|
provisioner.vm.box = "tinkerbelloss/sandbox-ubuntu1804" provisioner.vm.box = "tinkerbelloss/sandbox-ubuntu1804"
provisioner.vm.box_version = "0.1.0" provisioner.vm.box_version = "0.2.0"
provisioner.vm.hostname = 'provisioner' provisioner.vm.hostname = 'provisioner'
provisioner.vm.synced_folder './../../', '/vagrant' provisioner.vm.synced_folder './../../', '/vagrant'
provisioner.vm.provision :shell, provisioner.vm.provision :shell,

View File

@ -9,7 +9,8 @@ setup_docker() (
ca-certificates \ ca-certificates \
curl \ curl \
gnupg-agent \ gnupg-agent \
software-properties-common software-properties-common \
;
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | curl -fsSL https://download.docker.com/linux/ubuntu/gpg |
sudo apt-key add - sudo apt-key add -
@ -22,16 +23,24 @@ setup_docker() (
sudo add-apt-repository "$repo" sudo add-apt-repository "$repo"
sudo apt-get update sudo apt-get update
sudo apt-get install -y docker-ce docker-ce-cli containerd.io sudo apt-get install -y \
containerd.io \
docker-ce \
docker-ce-cli \
;
) )
# from https://docs.docker.com/compose/install/
setup_docker_compose() ( setup_docker_compose() (
# from https://docs.docker.com/compose/install/ local name url
sudo curl -L \ name=docker-compose-$(uname -s)-$(uname -m)
"https://github.com/docker/compose/releases/download/1.26.0/docker-compose-$(uname -s)-$(uname -m)" \ url=https://github.com/docker/compose/releases/download/1.26.0/$name
-o /usr/local/bin/docker-compose curl -fsSLO "$url"
curl -fsSLO "$url.sha256"
sudo chmod +x /usr/local/bin/docker-compose sha256sum -c <"$name.sha256"
rm -f "$name.sha256"
chmod +x "$name"
sudo mv "$name" /usr/local/bin/docker-compose
) )
main() ( main() (
@ -45,3 +54,4 @@ main() (
) )
main main
sync # do not remove!

View File

@ -34,12 +34,12 @@ setup_nat() (
main() ( main() (
export DEBIAN_FRONTEND=noninteractive export DEBIAN_FRONTEND=noninteractive
if [ ! -f ./.env ]; then if ! [[ -f ./.env ]]; then
./generate-envrc.sh eth1 >.env ./generate-env.sh eth1 >.env
fi fi
# shellcheck disable=SC1091 # shellcheck disable=SC1091
. ./.env source ./.env
make_certs_writable make_certs_writable
@ -51,6 +51,9 @@ main() (
secure_certs secure_certs
configure_vagrant_user configure_vagrant_user
set +x # don't want the stderr output from xtrace messing with the post-setup-message
[[ -f /tmp/post-setup-message ]] && cat /tmp/post-setup-message
) )
main main

62
docs/DCO.md Normal file
View File

@ -0,0 +1,62 @@
# DCO Sign Off
All authors to the project retain copyright to their work. However, to ensure
that they are only submitting work that they have rights to, we are requiring
everyone to acknowledge this by signing their work.
Since this signature indicates your rights to the contribution and
certifies the statements below, it must contain your real name and
email address. Various forms of noreply email address must not be used.
Any copyright notices in this repository should specify the authors as "The
project authors".
To sign your work, just add a line like this at the end of your commit message:
```text
Signed-off-by: Jess Owens <jowens@tinkerbell.org>
```
This can easily be done with the `--signoff` option to `git commit`.
By doing this you state that you can certify the following (from [https://developercertificate.org/][1]):
```text
Developer Certificate of Origin
Version 1.1
Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
1 Letterman Drive
Suite D4700
San Francisco, CA, 94129
Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.
Developer's Certificate of Origin 1.1
By making a contribution to this project, I certify that:
(a) The contribution was created in whole or in part by me and I
have the right to submit it under the open source license
indicated in the file; or
(b) The contribution is based upon previous work that, to the best
of my knowledge, is covered under an appropriate open source
license and I have the right under that license to submit that
work with modifications, whether created in whole or in part
by me, under the same open source license (unless I am
permitted to submit under a different license), as indicated
in the file; or
(c) The contribution was provided directly to me by some other
person who certified (a), (b) or (c) and I have not modified
it.
(d) I understand and agree that this project and the contribution
are public and that a record of the contribution (including all
personal information I submit with it, including my sign-off) is
maintained indefinitely and may be redistributed consistent with
this project or the open source license(s) involved.
```

110
generate-env.sh Executable file
View File

@ -0,0 +1,110 @@
#!/usr/bin/env bash
# stops the execution if a command or pipeline has an error
set -eu
if command -v tput >/dev/null && tput setaf 1 >/dev/null 2>&1; then
# color codes
RED="$(tput setaf 1)"
RESET="$(tput sgr0)"
fi
ERR="${RED:-}ERROR:${RESET:-}"
source ./current_versions.sh
err() (
if [[ -z ${1:-} ]]; then
cat >&2
else
echo "$ERR " "$@" >&2
fi
)
candidate_interfaces() (
ip -o link show |
awk -F': ' '{print $2}' |
sed 's/[ \t].*//;/^\(lo\|bond0\|\|\)$/d' |
sort
)
validate_tinkerbell_network_interface() (
local tink_interface=$1
if ! candidate_interfaces | grep -q "^$tink_interface$"; then
err "Invalid interface ($tink_interface) selected, must be one of:"
candidate_interfaces | err
return 1
else
return 0
fi
)
generate_password() (
head -c 12 /dev/urandom | sha256sum | cut -d' ' -f1
)
generate_env() (
local tink_interface=$1
validate_tinkerbell_network_interface "$tink_interface"
local tink_password
tink_password=$(generate_password)
local registry_password
registry_password=$(generate_password)
cat <<-EOF
# Tinkerbell Stack version
export OSIE_DOWNLOAD_LINK=${OSIE_DOWNLOAD_LINK}
export TINKERBELL_TINK_SERVER_IMAGE=${TINKERBELL_TINK_SERVER_IMAGE}
export TINKERBELL_TINK_CLI_IMAGE=${TINKERBELL_TINK_CLI_IMAGE}
export TINKERBELL_TINK_BOOTS_IMAGE=${TINKERBELL_TINK_BOOTS_IMAGE}
export TINKERBELL_TINK_HEGEL_IMAGE=${TINKERBELL_TINK_HEGEL_IMAGE}
export TINKERBELL_TINK_WORKER_IMAGE=${TINKERBELL_TINK_WORKER_IMAGE}
# Network interface for Tinkerbell's network
export TINKERBELL_NETWORK_INTERFACE="$tink_interface"
# Decide on a subnet for provisioning. Tinkerbell should "own" this
# network space. Its subnet should be just large enough to be able
# to provision your hardware.
export TINKERBELL_CIDR=${TINKERBELL_CIDR:-"29"}
# Host IP is used by provisioner to expose different services such as
# tink, boots, etc.
#
# The host IP should the first IP in the range, and the Nginx IP
# should be the second address.
export TINKERBELL_HOST_IP=${TINKERBELL_HOST_IP:-"192.168.1.1"}
# Tink server username and password
export TINKERBELL_TINK_USERNAME=admin
export TINKERBELL_TINK_PASSWORD="$tink_password"
# Docker Registry's username and password
export TINKERBELL_REGISTRY_USERNAME=admin
export TINKERBELL_REGISTRY_PASSWORD="$registry_password"
# Tink cli options
export TINKERBELL_GRPC_AUTHORITY=${TINKERBELL_HOST_IP:-"192.168.1.1"}:42113
export TINKERBELL_CERT_URL=http://${TINKERBELL_HOST_IP:-"192.168.1.1"}:42114/cert
# Legacy options, to be deleted:
export FACILITY=onprem
export ROLLBAR_TOKEN=ignored
export ROLLBAR_DISABLE=1
EOF
)
main() (
if [[ -z ${1:-} ]]; then
err "Usage: $0 network-interface-name > .env"
exit 1
fi
generate_env "$1"
)
main "$@"

View File

@ -1,105 +0,0 @@
#!/usr/bin/env bash
# stops the execution if a command or pipeline has an error
set -eu
if command -v tput >/dev/null && tput setaf 1 >/dev/null 2>&1; then
# color codes
RED="$(tput setaf 1)"
RESET="$(tput sgr0)"
fi
ERR="${RED:-}ERROR:${RESET:-}"
source ./current_versions.sh
err() (
if [ -z "${1:-}" ]; then
cat >&2
else
echo "$ERR " "$@" >&2
fi
)
candidate_interfaces() (
ip -o link show |
awk -F': ' '{print $2}' |
sed 's/[ \t].*//;/^\(lo\|bond0\|\|\)$/d' |
sort
)
validate_tinkerbell_network_interface() (
local tink_interface=$1
if ! candidate_interfaces | grep -q "^$tink_interface$"; then
err "Invalid interface ($tink_interface) selected, must be one of:"
candidate_interfaces | err
return 1
else
return 0
fi
)
generate_password() (
head -c 12 /dev/urandom | sha256sum | cut -d' ' -f1
)
generate_envrc() (
local tink_interface=$1
validate_tinkerbell_network_interface "$tink_interface"
local tink_password
tink_password=$(generate_password)
local registry_password
registry_password=$(generate_password)
cat <<EOF
# Tinkerbell Stack version
export OSIE_DOWNLOAD_LINK=${OSIE_DOWNLOAD_LINK}
export TINKERBELL_TINK_SERVER_IMAGE=${TINKERBELL_TINK_SERVER_IMAGE}
export TINKERBELL_TINK_CLI_IMAGE=${TINKERBELL_TINK_CLI_IMAGE}
export TINKERBELL_TINK_BOOTS_IMAGE=${TINKERBELL_TINK_BOOTS_IMAGE}
export TINKERBELL_TINK_HEGEL_IMAGE=${TINKERBELL_TINK_HEGEL_IMAGE}
export TINKERBELL_TINK_WORKER_IMAGE=${TINKERBELL_TINK_WORKER_IMAGE}
# Network interface for Tinkerbell's network
export TINKERBELL_NETWORK_INTERFACE="$tink_interface"
# Decide on a subnet for provisioning. Tinkerbell should "own" this
# network space. Its subnet should be just large enough to be able
# to provision your hardware.
export TINKERBELL_CIDR=29
# Host IP is used by provisioner to expose different services such as
# tink, boots, etc.
#
# The host IP should the first IP in the range, and the Nginx IP
# should be the second address.
export TINKERBELL_HOST_IP=192.168.1.1
# Tink server username and password
export TINKERBELL_TINK_USERNAME=admin
export TINKERBELL_TINK_PASSWORD="$tink_password"
# Docker Registry's username and password
export TINKERBELL_REGISTRY_USERNAME=admin
export TINKERBELL_REGISTRY_PASSWORD="$registry_password"
# Legacy options, to be deleted:
export FACILITY=onprem
export ROLLBAR_TOKEN=ignored
export ROLLBAR_DISABLE=1
EOF
)
main() (
if [ -z "${1:-}" ]; then
err "Usage: $0 network-interface-name > .env"
exit 1
fi
generate_envrc "$1"
)
main "$@"

View File

@ -1,7 +1,7 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# stops the execution if a command or pipeline has an error # stops the execution if a command or pipeline has an error
set -eu set -euxo pipefail
# Tinkerbell stack Linux setup script # Tinkerbell stack Linux setup script
# #
@ -38,7 +38,7 @@ NEXT="${GREEN:-}NEXT:${RESET:-}"
get_distribution() ( get_distribution() (
local lsb_dist="" local lsb_dist=""
# Every system that we officially support has /etc/os-release # Every system that we officially support has /etc/os-release
if [ -r /etc/os-release ]; then if [[ -r /etc/os-release ]]; then
# shellcheck disable=SC1091 # shellcheck disable=SC1091
lsb_dist="$(. /etc/os-release && echo "$ID")" lsb_dist="$(. /etc/os-release && echo "$ID")"
fi fi
@ -50,7 +50,7 @@ get_distribution() (
get_distro_version() ( get_distro_version() (
local lsb_version="0" local lsb_version="0"
# Every system that we officially support has /etc/os-release # Every system that we officially support has /etc/os-release
if [ -r /etc/os-release ]; then if [[ -r /etc/os-release ]]; then
# shellcheck disable=SC1091 # shellcheck disable=SC1091
lsb_version="$(. /etc/os-release && echo "$VERSION_ID")" lsb_version="$(. /etc/os-release && echo "$VERSION_ID")"
fi fi
@ -110,6 +110,18 @@ setup_networking() (
else else
echo "$ERR tinkerbell network interface configuration failed" echo "$ERR tinkerbell network interface configuration failed"
fi fi
NAT_INTERFACE=""
if [[ -r .nat_interface ]]; then
NAT_INTERFACE=$(cat .nat_interface)
fi
if [[ -n $NAT_INTERFACE ]] && ip addr show "$NAT_INTERFACE" &>/dev/null; then
# TODO(nshalman) the terraform code would just run these commands as-is once
# but it would be nice to make these more persistent based on OS
iptables -A FORWARD -i "$TINKERBELL_NETWORK_INTERFACE" -o "$NAT_INTERFACE" -j ACCEPT
iptables -A FORWARD -i "$NAT_INTERFACE" -o "$TINKERBELL_NETWORK_INTERFACE" -m state --state ESTABLISHED,RELATED -j ACCEPT
iptables -t nat -A POSTROUTING -o "$NAT_INTERFACE" -j MASQUERADE
fi
) )
setup_networking_manually() ( setup_networking_manually() (
@ -123,10 +135,10 @@ setup_networking_manually() (
setup_network_forwarding() ( setup_network_forwarding() (
# enable IP forwarding for docker # enable IP forwarding for docker
if [ "$(sysctl -n net.ipv4.ip_forward)" != "1" ]; then if (($(sysctl -n net.ipv4.ip_forward) != 1)); then
if [ -d /etc/sysctl.d ]; then if [[ -d /etc/sysctl.d ]]; then
echo "net.ipv4.ip_forward=1" >/etc/sysctl.d/99-tinkerbell.conf echo "net.ipv4.ip_forward=1" >/etc/sysctl.d/99-tinkerbell.conf
elif [ -f /etc/sysctl.conf ]; then elif [[ -f /etc/sysctl.conf ]]; then
echo "net.ipv4.ip_forward=1" >>/etc/sysctl.conf echo "net.ipv4.ip_forward=1" >>/etc/sysctl.conf
fi fi
@ -159,7 +171,7 @@ setup_networking_netplan() (
) )
setup_networking_ubuntu_legacy() ( setup_networking_ubuntu_legacy() (
if [ ! -f /etc/network/interfaces ]; then if ! [[ -f /etc/network/interfaces ]]; then
echo "$ERR file /etc/network/interfaces not found" echo "$ERR file /etc/network/interfaces not found"
exit 1 exit 1
fi fi
@ -212,7 +224,7 @@ EOF
local cfgfile="/etc/sysconfig/network-scripts/ifcfg-$TINKERBELL_NETWORK_INTERFACE" local cfgfile="/etc/sysconfig/network-scripts/ifcfg-$TINKERBELL_NETWORK_INTERFACE"
if [ -f "$cfgfile" ]; then if [[ -f $cfgfile ]]; then
echo "$ERR network config already exists: $cfgfile" echo "$ERR network config already exists: $cfgfile"
echo "$BLANK Please update it to match this configuration:" echo "$BLANK Please update it to match this configuration:"
echo "$content" echo "$content"
@ -233,12 +245,12 @@ setup_osie() (
local osie_current=$STATEDIR/webroot/misc/osie/current local osie_current=$STATEDIR/webroot/misc/osie/current
local tink_workflow=$STATEDIR/webroot/workflow/ local tink_workflow=$STATEDIR/webroot/workflow/
if [ ! -d "$osie_current" ] || [ ! -d "$tink_workflow" ]; then if [[ ! -d $osie_current ]] || [[ ! -d $tink_workflow ]]; then
mkdir -p "$osie_current" mkdir -p "$osie_current"
mkdir -p "$tink_workflow" mkdir -p "$tink_workflow"
pushd "$SCRATCH" pushd "$SCRATCH"
if [ -z "${TB_OSIE_TAR:-}" ]; then if [[ -z ${TB_OSIE_TAR:-} ]]; then
curl "${OSIE_DOWNLOAD_LINK}" -o ./osie.tar.gz curl "${OSIE_DOWNLOAD_LINK}" -o ./osie.tar.gz
tar -zxf osie.tar.gz tar -zxf osie.tar.gz
else else
@ -293,7 +305,7 @@ check_container_status() (
--filter "event=health_status" \ --filter "event=health_status" \
--format '{{.Status}}') --format '{{.Status}}')
if [ "$status" != "health_status: healthy" ]; then if [[ $status != "health_status: healthy" ]]; then
echo "$ERR $container_name is not healthy. status: $status" echo "$ERR $container_name is not healthy. status: $status"
exit 1 exit 1
fi fi
@ -302,7 +314,7 @@ check_container_status() (
generate_certificates() ( generate_certificates() (
mkdir -p "$STATEDIR/certs" mkdir -p "$STATEDIR/certs"
if [ ! -f "$STATEDIR/certs/ca.json" ]; then if ! [[ -f "$STATEDIR/certs/ca.json" ]]; then
jq \ jq \
'. '.
| .names[0].L = $facility | .names[0].L = $facility
@ -313,7 +325,7 @@ generate_certificates() (
>"$STATEDIR/certs/ca.json" >"$STATEDIR/certs/ca.json"
fi fi
if [ ! -f "$STATEDIR/certs/server-csr.json" ]; then if ! [[ -f "$STATEDIR/certs/server-csr.json" ]]; then
jq \ jq \
'. '.
| .hosts += [ $ip, "tinkerbell.\($facility).packet.net" ] | .hosts += [ $ip, "tinkerbell.\($facility).packet.net" ]
@ -335,13 +347,13 @@ generate_certificates() (
local certs_dir="/etc/docker/certs.d/$TINKERBELL_HOST_IP" local certs_dir="/etc/docker/certs.d/$TINKERBELL_HOST_IP"
# copy public key to NGINX for workers # copy public key to NGINX for workers
if ! cmp --quiet "$STATEDIR"/certs/ca.pem "$STATEDIR/webroot/workflow/ca.pem"; then if ! cmp --quiet "$STATEDIR/certs/ca.pem" "$STATEDIR/webroot/workflow/ca.pem"; then
cp "$STATEDIR"/certs/ca.pem "$STATEDIR/webroot/workflow/ca.pem" cp "$STATEDIR/certs/ca.pem" "$STATEDIR/webroot/workflow/ca.pem"
fi fi
# update host to trust registry certificate # update host to trust registry certificate
if ! cmp --quiet "$STATEDIR/certs/ca.pem" "$certs_dir/tinkerbell.crt"; then if ! cmp --quiet "$STATEDIR/certs/ca.pem" "$certs_dir/tinkerbell.crt"; then
if [ ! -d "$certs_dir/tinkerbell.crt" ]; then if ! [[ -d "$certs_dir/" ]]; then
# The user will be told to create the directory # The user will be told to create the directory
# in the next block, if copying the certs there # in the next block, if copying the certs there
# fails. # fails.
@ -351,7 +363,7 @@ generate_certificates() (
echo "$ERR please copy $STATEDIR/certs/ca.pem to $certs_dir/tinkerbell.crt" echo "$ERR please copy $STATEDIR/certs/ca.pem to $certs_dir/tinkerbell.crt"
echo "$BLANK and run $0 again:" echo "$BLANK and run $0 again:"
if [ ! -d "$certs_dir" ]; then if ! [[ -d $certs_dir ]]; then
echo "sudo mkdir -p '$certs_dir'" echo "sudo mkdir -p '$certs_dir'"
fi fi
echo "sudo cp '$STATEDIR/certs/ca.pem' '$certs_dir/tinkerbell.crt'" echo "sudo cp '$STATEDIR/certs/ca.pem' '$certs_dir/tinkerbell.crt'"
@ -394,7 +406,7 @@ bootstrap_docker_registry() (
setup_docker_registry() ( setup_docker_registry() (
local registry_images="$STATEDIR/registry" local registry_images="$STATEDIR/registry"
if [ ! -d "$registry_images" ]; then if ! [[ -d $registry_images ]]; then
mkdir -p "$registry_images" mkdir -p "$registry_images"
fi fi
start_registry start_registry
@ -415,13 +427,15 @@ command_exists() (
) )
check_command() ( check_command() (
if command_exists "$1"; then if ! command_exists "$1"; then
echo "$BLANK Found prerequisite: $1" echo "$ERR Prerequisite executable command not found: $1"
return 0
else
echo "$ERR Prerequisite command not installed: $1"
return 1 return 1
fi fi
if ! [[ -s "$(which "$1")" ]]; then
echo "$ERR Prerequisite command is an empty file: $1"
fi
echo "$BLANK Found prerequisite: $1"
return 0
) )
check_prerequisites() ( check_prerequisites() (
@ -457,15 +471,15 @@ check_prerequisites() (
;; ;;
esac esac
if [ $failed -eq 1 ]; then if ((failed == 1)); then
echo "$ERR Prerequisites not met. Please install the missing commands and re-run $0." echo "$ERR Prerequisites not met. Please install the missing commands and re-run $0."
exit 1 exit 1
fi fi
) )
whats_next() ( whats_next() (
echo "$NEXT 1. Enter /vagrant/deploy and run: source ../.env; docker-compose up -d" echo "$NEXT 1. Enter /deploy and run: source ../.env; docker-compose up -d"
echo "$BLANK 2. Try executing your fist workflow." echo "$BLANK 2. Try executing your first workflow."
echo "$BLANK Follow the steps described in https://tinkerbell.org/examples/hello-world/ to say 'Hello World!' with a workflow." echo "$BLANK Follow the steps described in https://tinkerbell.org/examples/hello-world/ to say 'Hello World!' with a workflow."
) )
@ -477,21 +491,23 @@ do_setup() (
echo "$INFO starting tinkerbell stack setup" echo "$INFO starting tinkerbell stack setup"
check_prerequisites "$lsb_dist" "$lsb_version" check_prerequisites "$lsb_dist" "$lsb_version"
if [ ! -f "$ENV_FILE" ]; then if ! [[ -f $ENV_FILE ]]; then
echo "$ERR Run './generate-envrc.sh network-interface > \"$ENV_FILE\"' before continuing." echo "$ERR Run './generate-env.sh network-interface > \"$ENV_FILE\"' before continuing."
exit 1 exit 1
fi fi
# shellcheck disable=SC1090 # shellcheck disable=SC1090
source "$ENV_FILE" source "$ENV_FILE"
if [[ -z $TINKERBELL_SKIP_NETWORKING ]]; then
setup_networking "$lsb_dist" "$lsb_version" setup_networking "$lsb_dist" "$lsb_version"
fi
setup_osie setup_osie
generate_certificates generate_certificates
setup_docker_registry setup_docker_registry
echo "$INFO tinkerbell stack setup completed successfully on $lsb_dist server" echo "$INFO tinkerbell stack setup completed successfully on $lsb_dist server"
whats_next whats_next | tee /tmp/post-setup-message
) )
# wrapped up in a function so that we have some protection against only getting # wrapped up in a function so that we have some protection against only getting

View File

@ -14,5 +14,5 @@ in
with pkgs; with pkgs;
mkShell { mkShell {
buildInputs = [ go nodePackages.prettier shellcheck shfmt terraform gpgme packer vagrant ]; buildInputs = [ go nodePackages.prettier jq shellcheck shfmt terraform_0_14 gpgme packer vagrant ];
} }