Falco — Security at Runtime for Kubernetes

Runtime attacks occur in the early stages of devastating attacks!

Kubernetes - Security

Kubernetes offers relatively little in the way of security tooling; it doesn’t deliver security monitoring or threat detection capabilities. It expects you — the DevSecOps guru — to observe for and respond to security-related matters yourself.

In this post, we’ll provide a context on:

  • What is Runtime Security?
  • Why & How Falco can assist DevSecOps teams in securing their environment?
  • Architecture & Core Components of Falco: How to build a safe and resilient infrastructure against unwanted activities in application workloads?
  • Deploy: How to customize Falco deployment for seamless integration with our ecosystem.

Runtime Security

Several security barriers could be implemented, helping create a less risky environment, But still, not enough! One can do better by adding/enforcing a closer security layer. This is possible using “Runtime Security,”; A valuable mechanism to granularly scope what workloads run on a workspace, how they run, and enforce quick response to unexpected or unwanted changes.

Runtime Security: Yet another Security layer to proactively protect workloads!

Why Falco?

A “Frequent change” is a fundamental part of the cloud-native nowadays, continuously delivering/deploying software and infra stacks. Unfortunately, anomalous events may happen, affecting the security of a system and to which one must respond accurately. It’s not practical nor reasonable to tail the logs by hand in order to detect potential risks (hundreds/thousands of API requests each minute!!).

Falco — High-Level Overview
  • Filtering network traffic and taking action on established connections,
  • Detecting configuration drift and taking action upon any unexpected changes on a FileSystem,
  • Detect misbehavior such as unauthorized privilege-gains, and react to any unwanted binary execution,
  • and much more could be accomplished with Falco.

Architecture & Core Components

Falco is a behavioral activity monitoring tool built explicitly for containers.

  1. Events (introduced by Kernel, SysCalls interceptions, ..)
  2. Powerful Rules Engine where the stream of events is asserted
  3. And Alerts are triggered when a rule is violated

Components

  • Events/Inputs: Provides support with several types of input that vary from the powerful Syscalls, Kubernetes AuditLogs, and Cloud AuditLogs. All gathered through libscap/libinsp libraries.
  • Rules Engine/Processing: A flexible Rules Filter engine allows you to describe any host or container behavior or activity. (Runs as Kubernetes DaemonSets)
  • Rules Configuration: It enables full detection of suspicious activity defined by a set of rules loaded from a Rules-configuration. (Loaded from Kubernetes Secrets)
  • Alerts/Output: Reduce risk via immediate alerts through Falco-Sidekick to an AlertManager, Pager Duty, and more

Deploy & Configure

For better isolation, the Falco community highly recommends running Falco outside of the container runtimes as a service in systemd; However, this could not be conceivable if running on a managed Kubernetes environment. The alternative is to get it up and running as privileged DaemonSets, to monitor all kinds of System events captured from each and every node part of the Kubernetes Cluster.

Root-level access is required, as Falco still needs to hook into the Kernel on the host

It is strongly advised to rely on Kubernetes QoS and Priority Classes to configure Falco well, so it keeps available and schedulable in the most challenging times that a Kubernetes cluster could face (degraded, short in resources, etc..)

Please refer to my previous blog-post for “K8s Cluster Protection — PriorityClasses” if you require additional details.

To start a custom deployment of Falco, we will need to download/modify the chart Values file. Before we start, we should add the Falco repo/chart:

helm repo add falcosecurity https://falcosecurity.github.io/charts
helm repo update
helm get values falcosecurity -o yaml > values.yaml
vim values.yaml
falco:
# Configure JSON as Output format
jsonOutput: true
jsonIncludeOutputProperty: true
falco.jsonIncludeTagsProperty: true
# Activates the K8s Audit Log feature for Falco
auditLog:
enabled: true
falcosidekick:
enabled: true
config:
existingSecret: "falco-sidekick-outputchannels-secrets"
webui:
enabled: enabled
helm upgrade --install falco falcosecurity/falco -n falco-system
> kubectl logs deployment/falco-falcosidekick -n falco-system
Found 2 pods, using pod/falco-falcosidekick-5c797f56c5-8zfbg
2022/03/08 09:32:23 [INFO] : Enabled Outputs : [WebUI]
2022/03/08 09:32:23 [INFO] : Falco Sidekick is listening on :2801
2022/03/08 09:32:51 [INFO] : WebUI - Post OK (200)

Falco Rules

Falco comes with a reach rule-set particularly built to match against the main compliance frameworks, such as and not limited to PCI-DSS. (As an example, several rules are tagged with PCI offer a specific coverage to the PCI-DSS framework)

Rules are asserted at runtime; When a rule is violated on a system, Falco triggers alerts related to the violation.

Falco Alerts

Out of the box, Falco provides a very limited set of output channels: {stdout, file, Syslog, spawned program, HTTP endpoint, and a gRPC API}. While this could be enough for certain use-cases, outputs are easily extendable with Falco-Sidekick. The latter offers broad integration with tools from ChatOps, Monitoring/Alerting solutions, UI, etc.

Time to play!! Simulate a malicious activity!

Let’s get into the fun part and try to modify a running container (in 3 easy steps).

kubectl port-forward svc/falco-falcosidekick-ui -n falco-system 2802
kubectl run -it malicious --image=falcosecurity/falco-builder \
-n falco-system -- bash
#include <stdio.h>
#include <stdlib.h>

int main() {
char sentence[1000];

// creating file pointer to work with files
FILE *fptr;

// opening file in writing mode
fptr = fopen("program.txt", "w");

// exiting program
if (fptr == NULL) {
printf("Error!");
exit(1);
}
printf("Enter a sentence:\n");
fgets(sentence, sizeof(sentence), stdin);
fprintf(fptr, "%s", sentence);
fclose(fptr);
return 0;
}
gcc malicious.c -o sim-hack
./sim-hack
15:22:02.817540337: Error File below / or /root opened for writing (user=root user_loginuid=-1 command=hack parent=bash file=/program.txt program=hack container_id=c59fb04043ff image=falcosecurity/falco-builder) k8s.ns=falco-system k8s.pod=malicious container=c59fb04043ff k8s.ns=falco-system k8s.pod=malicious container=c59fb04043ff k8s.ns=falco-system k8s.pod=malicious container=c59fb04043ff
Falco-SideKick-UI — Suspicious activity

Falco — Event Generator

We can merely run a Falco Event Generator image to initiate generating dummy events so that you can watch on Falco:

kubectl run --it falco-event-generator \
--image=falcosecurity/falco-event-generator -n falco-system

Falco Playground — Rules

https://falco.org/labs/

Falco —Self Security Report / Vulnerabilities !!

Falco itself contains several known vulnerabilities; More than 690 vulnerabilities are identified in its binaries, and we hope to see them resolved very soon.

To Conclude

In this post, we scratched the surface of possibilities; We learned how to achieve continuous Runtime-Security monitoring for container-based workloads running on Kubernetes through custom integration between Falco, Falco-SideKick, WebUI, and AWS CloudWatch/PagerDuty.

Thank you so much for reading me!

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Aymen Abdelwahed

Aymen Abdelwahed

Is a Cloud-Native enthusiast with 12 plus years of experience. He’s continuously immersing himself in the latest technology trends & projects.