Working with SUSE CaaSP
This gives you the opportunity to utilize integration of SUSE CaaSP with Terraform, meaning: no more hardcoded IP Addresses for your tests!
To do that Management Node where you have the Terraform configuration of your cluster lying around. Here's what it expected:
- a SLES15-SP1 infrastructure ready to be deployed using
skuba - a running SSH-Agent loaded with the keys injected into your nodes.
- a terminal where you can run the dudenetes commands (ssh-agent must be running there)
-
kubectlandhelmare also required as you will use them a lot in your tests
In that case Dudenetes assumes you have an appropriate kubeconfig file to talk with your cluster. If you don't have a cluster yet, but only the infrastructure, then it is expected to hardcode the IP addresses of loadbalancer, masters and workers in your *.feature files.
- Know the IP Addresses of your nodes (if you want to talk to them specifically)
- Have a working
kubeconfig - Have
kubectlandhelminstalled as you will probably use them a lot in your tests
I assume you already have this, so don't bother if you do.
sudo zypper install go
If you haven't setup Go ever in your life, you might want to do something like this in your ~/.bashrc:
GOPATH=$(go env | grep GOPATH | awk -F '"' '{ print $2 }')
GOROOT=$(go env | grep GOROOT | awk -F '"' '{ print $2 }')
export PATH=$PATH:$GOPATH/bin:$GOROOT/binAnd then run source ~/.bashrc to activate the changes.
If your Go stuff are properly configured, you should be able to just run these:
go get -v github.com/DATA-DOG/godog/cmd/godog
go get -v github.com/joho/godotenv
go get -v github.com/drpaneas/dudenetesBy now you should be able to run Dudenetes:
root@skyrim:/# dudenetes
The most relaxed testing framework of Kubernetes in the world
Usage:
dudenetes [command]
Available Commands:
help Help about any command
list Lists the tests in the current directory
test Executes a test scenario
Flags:
--config string config file (default is $HOME/.dudenetes.yaml)
-h, --help help for dudenetes
-t, --toggle Help message for toggle
Use "dudenetes [command] --help" for more information about a command.
subcommand is requiredFor this example, we will put everything under a folder called tests:
mkdir tests; cd tests
vi helloworld.feature
root@skyrim:/tests# cat helloworld.feature
Feature: Hello World
Bootstrapping the cluster is the initial process of starting up the cluster
Here you can write whate you want. It is a description your test without
any specific language. It's just text in a sense of a summary or description.
Scenario: Run your first test dude
When you type "touch file"
Then a file must be created "ls | grep file"
Dudenetes is going to generate the code for this feature so you can test it. For starters, try to run this file:
Feature: Hello World
Bootstrapping the cluster is the initial process of starting up the cluster
Here you can write whate you want. It is a description your test without
any specific language. It's just text in a sense of a summary or description.
Scenario: Run your first test dude # helloworld.feature:7
When you type "touch file"
Then a file must be created "ls | grep file"
1 scenarios (1 undefined)
2 steps (2 undefined)
67.555µs
You can implement step definitions for undefined steps with these snippets:
func youType(arg1 string) error {
return godog.ErrPending
}
func aFileMustBeCreated(arg1 string) error {
return godog.ErrPending
}
func FeatureContext(s *godog.Suite) {
s.Step(`^you type "([^"]*)"$`, youType)
s.Step(`^a file must be created "([^"]*)"$`, aFileMustBeCreated)
}This is the boilerplate code you need. Now, create a helloworld_test.go file and copy-paste that code, along with the absolute basic for Go to compile it:
package main
import (
"github.com/DATA-DOG/godog"
)
func youType(arg1 string) error {
return godog.ErrPending
}
func aFileMustBeCreated(arg1 string) error {
return godog.ErrPending
}
func FeatureContext(s *godog.Suite) {
s.Step(`^you type "([^"]*)"$`, youType)
s.Step(`^a file must be created "([^"]*)"$`, aFileMustBeCreated)
}Now try to run the test again:
root@skyrim:/tests# dudenetes test -f helloworld.feature
Feature: Hello World
Bootstrapping the cluster is the initial process of starting up the cluster
Here you can write whate you want. It is a description your test without
any specific language. It's just text in a sense of a summary or description.
Scenario: Run your first test dude # helloworld.feature:7
When you type "touch file" # helloworld_test.go:8 -> youType
TODO: write pending definition
Then a file must be created "ls | grep file" # helloworld_test.go:12 -> aFileMustBeCreated
1 scenarios (1 pending)
2 steps (1 pending, 1 skipped)
184.545µsNotice that this time it ran the tests! It says pending because all of your test-functions are returning return godog.ErrPending by default. In the next section we are going to modify the tests.
Open again the helloworld_test.go file and this time modify the test to execute the arguments striped from the helloworld.feature file:
package main
import (
"github.com/DATA-DOG/godog"
"github.com/drpaneas/dudenetes/pkg/run"
)
func youType(arg1 string) error {
output, err := run.Cmd(arg1)
if err != nil {
return run.LogError(arg1, output, err)
}
return nil
}
func aFileMustBeCreated(arg1 string) error {
output, err := run.CmdWithPipes(arg1) // Use this if the command has pipes
if err != nil {
return run.LogError(arg1, output, err)
}
return nil
}
func FeatureContext(s *godog.Suite) {
s.Step(`^you type "([^"]*)"$`, youType)
s.Step(`^a file must be created "([^"]*)"$`, aFileMustBeCreated)
}Run again the test and see the results:
Feature: Hello World
Bootstrapping the cluster is the initial process of starting up the cluster
Here you can write whate you want. It is a description your test without
any specific language. It's just text in a sense of a summary or description.
Scenario: Run your first test dude # helloworld.feature:7
When you type "touch file" # helloworld_test.go:8 -> youType
Then a file must be created "ls | grep file" # helloworld_test.go:16 -> aFileMustBeCreated
1 scenarios (1 passed)
2 steps (2 passed)
3.733132msThe output is automatically saved into a file called helloworld.feature.log so you don't have to worry to redirect the output.
In this example we are going to run dudenetes from the Management Node. Over there we have already run terraform apply -auto-approve so the infrastructure is up and running, but no cluster yet. We make sure that ssh-agent is running and on the terminal we are using, so let's go:
Feature: Bootstraping
Bootstrapping the cluster is the initial process of starting up the cluster
and defining which of the nodes are masters and which workers.
For maximum automation of this process SUSE CaaS Platform uses the skuba package.
The tests assumes you have skuba already available in your machine
and you have already deployed the requred infrastructure along with
the SSH-agent running from the terminal you are issuing dudenetes commands.
Scenario: Initialize the cluster
Given you have deployed the required infrastructure for SUSE CaaSP
When you do "skuba cluster init --control-plane $loadbalancer my-cluster"
Then "my-cluster" dir should be created containing the IP of the loadbalancer "grep -r $loadbalancer my-cluster"
Scenario: Bootstrap the master node
Given you run "skuba -v 5 node bootstrap --user sles --sudo --target $master1 master-1" with a timeout of 500 seconds
And after configuring your new kubeconfig into this "cp admin.conf $HOME/.kube/config"
Then the master must be ready within 500 seconds timeout "kubectl get nodes | grep master-1 | grep --invert-match NotReady | grep Ready"
Scenario: Join the workers
When you run skuba node join "skuba -v 5 node join --role worker --user sles --sudo --target $worker1 worker-1" with 500 sec timeout
Then the node should be ready "kubectl get nodes | grep worker-1 | grep --invert-match NotReady | grep Ready" within 180 sec
When you run skuba node join "skuba -v 5 node join --role worker --user sles --sudo --target $worker2 worker-2" with 500 sec timeout
Then the node should be ready "kubectl get nodes | grep worker-2 | grep --invert-match NotReady | grep Ready" within 180 sec
When you run skuba node join "skuba -v 5 node join --role worker --user sles --sudo --target $worker3 worker-3" with 500 sec timeout
Then the node should be ready "kubectl get nodes | grep worker-3 | grep --invert-match NotReady | grep Ready" within 180 secpackage main
import (
"os"
"strings"
"github.com/DATA-DOG/godog"
"github.com/drpaneas/dudenetes/pkg/run"
"github.com/drpaneas/dudenetes/pkg/skuba"
)
var folder string
func youHaveDeployedTheRequiredInfrastructureForSUSECaaSP() error {
// Load the current Terraform output of the working cluster
err := skuba.LoadTF()
if err != nil {
return err
}
// Verify you have what you need
err = skuba.Need(1, 1, 3) // LB, Masters, Workers
if err != nil {
return err
}
return nil
}
func youDo(arg1 string) error {
arg1 = skuba.ReplaceVarsWithEnvs(arg1)
output, err := run.Cmd(arg1)
if err != nil {
return run.LogError(arg1, output, err)
}
return nil
}
func dirShouldBeCreatedContainingTheIPOfTheLoadbalancer(arg1, arg2 string) error {
folder = arg1
arg2 = skuba.ReplaceVarsWithEnvs(arg2)
output, err := run.Cmd(arg2)
if err != nil {
return run.LogError(arg2, output, err)
}
return nil
}
func youRunWithATimeoutOfSeconds(arg1 string, arg2 int) error {
arg1 = skuba.ReplaceVarsWithEnvs(arg1)
output, err := run.SlowCmdDir(arg1, arg2, folder)
if err != nil {
return run.LogError(arg1, output, err)
}
return nil
}
func afterConfiguringYourNewKubeconfigIntoThis(arg1 string) error {
arg1 = strings.ReplaceAll(arg1, "$HOME", os.Getenv("HOME"))
output, err := run.SlowCmdDir(arg1, 5, folder)
if err != nil {
return run.LogError(arg1, output, err)
}
return nil
}
func theMasterMustBeReadyWithinSecondsTimeout(arg1 int, arg2 string) error {
output, err := run.CmdRetry(arg2, arg1)
if err != nil {
return run.LogError(arg2, output, err)
}
return nil
}
func youRunSkubaNodeJoinWithSecTimeout(arg1 string, arg2 int) error {
arg1 = skuba.ReplaceVarsWithEnvs(arg1)
output, err := run.SlowCmdDir(arg1, arg2, folder)
if err != nil {
return run.LogError(arg1, output, err)
}
return nil
}
func theNodeShouldBeReadyWithinSec(arg1 string, arg2 int) error {
output, err := run.CmdRetry(arg1, arg2)
if err != nil {
return run.LogError(arg1, output, err)
}
return nil
}
func FeatureContext(s *godog.Suite) {
s.Step(`^you have deployed the required infrastructure for SUSE CaaSP$`, youHaveDeployedTheRequiredInfrastructureForSUSECaaSP)
s.Step(`^you do "([^"]*)"$`, youDo)
s.Step(`^"([^"]*)" dir should be created containing the IP of the loadbalancer "([^"]*)"$`, dirShouldBeCreatedContainingTheIPOfTheLoadbalancer)
s.Step(`^you run "([^"]*)" with a timeout of (\d+) seconds$`, youRunWithATimeoutOfSeconds)
s.Step(`^after configuring your new kubeconfig into this "([^"]*)"$`, afterConfiguringYourNewKubeconfigIntoThis)
s.Step(`^the master must be ready within (\d+) seconds timeout "([^"]*)"$`, theMasterMustBeReadyWithinSecondsTimeout)
s.Step(`^you run skuba node join "([^"]*)" with (\d+) sec timeout$`, youRunSkubaNodeJoinWithSecTimeout)
s.Step(`^the node should be ready "([^"]*)" within (\d+) sec$`, theNodeShouldBeReadyWithinSec)
}dudenetes test -f boostraping.feature -s ~/go/src/github.com/SUSE/skuba/ci/infra/vmware/This is going to look like that:
See larger: https://raw.githubusercontent.com/drpaneas/dudenetes/master/img/dudenetes_new_feature.png
