# Deployment Options

Streaming SQL processing of real-time data can run in 2 modes:

* SQL in Process
* SQL on Kubernetes

### SQL in Process  <a href="#sql-in-process" id="sql-in-process"></a>

Streaming SQL applications will be run as part of Lenses’ process, sharing resources, memory, and CPU time with the rest of the platform.

Keep in mind<br>

This mode of operation is meant to be used for development only.

As such, Lenses will not allow the creation of more than 50 SQL Processors in *In Process* mode, as this could impact the platform's stability and performance negatively.

For production, use the `KUBERNETES` mode for maximum flexibility and scalability.

Set the execution configuration to `IN_PROC`

```bash
# Set up Lenses SQL processing engine
lenses.sql.execution.mode = "IN_PROC"
```

Set the directory to store the internal state of the SQL Processors:

```bash
lenses.sql.state.dir = "/tmp/lenses-sql-kstream-state"
```

### Connections management  <a href="#connections-management" id="connections-management"></a>

Since Lenses 4.0, connection details are treated as secrets and stored in the internal database. That change allows Lenses to bring governance to the connections and subsequently, to the systems that Processors connect to.

SQL processors use the same connection details that Lenses uses to speak to Kafka and Schema Registry. The following properties are mounted, if present, on the file system for each processor:

* **Kafka**
  1. SSLTruststore
  2. SSLKeystore
* **Schema Registry**
  1. SSL Keystore
  2. SSL Truststore

The files structure created by applications is the following: `/run/[lenses_installation_id]/applications/`

Keep in mind\
Lenses require an installation folder with write permissions. The following are tried:

1. /run
2. /tmp

### SQL on Kubernetes  <a href="#sql-on-kubernetes" id="sql-on-kubernetes"></a>

Kubernetes can be used to deploy SQL Processors. To configure Kubernetes set the mode to `KUBERNETES` and configure the location of the kubeconfig file.

When Lenses is deployed inside Kubernetes, the `lenses.kubernetes.config.file` configuration entry should be set to an empty string. The Kubernetes client will auto-configure from the pod it is deployed in.

The streaming SQL docker image live in Dockerhub.

```bash
lenses.sql.execution.mode = KUBERNETES
# kubernetes configuration
lenses.kubernetes.config.file = "/home/lenses/.kube/config"
lenses.kubernetes.service.account = "default"
#lenses.kubernetes.processor.image.name = "" # Only needed if you use a custom image
#lenses.kubernetes.processor.image.tag = ""  # Only needed if you use a custom image

# Only needed if you want to tune the buffer size for incoming events from Kubernetes
#lenses.deployments.errors.buffer.size = 1000

# Only needed if you want to tune the buffer size for incoming errors from Kubernetes WS communication
#lenses.deployments.events.buffer.size = 10000
```

### Kerberos  <a href="#kerberos" id="kerberos"></a>

If Kerberos is required set the JAAS file for the SQL processors. Use the following configuration:

```bash
   lenses.kubernetes.processor.kafka.settings.security.protocol = SASL_PLAINTEXT
   lenses.kubernetes.processor.jaas                  = "/jaas-processors.conf"
   lenses.kubernetes.processor.kafka.settings.keytab = "/processor.keytab"
   lenses.kubernetes.processor.krb5                  = "/etc/krb5.conf"
```

<br>

For the jaas.conf, replace the paths so that the keytab points to `/mnt/secrets/kafka/keytab` the contents must be mounted in the Pods in a well-known location.

```bash
KafkaClient {
  com.sun.security.auth.module.Krb5LoginModule required
  useKeyTab=true
  keyTab="/mnt/secrets/kafka/keytab"
  storeKey=true
  useTicketCache=false
  serviceName="kafka"
  principal="principal@MYREALM";
};

/*
  Optional section for authentication to zookeeper
  Please also remember to set lenses.zookeeper.security.enabled=true
*/
Client {
  com.sun.security.auth.module.Krb5LoginModule required
   useKeyTab=true
   keyTab="/mnt/secrets/kafka/keytab"
   storeKey=true
   useTicketCache=false
   principal="principal@MYREALM";
};
```

<br>

Krb5 files cannot include the *includeDir* property as this will not resolve in the Pods. Extend the SQL Processor image and update Lenses to use your custom image if you require this.

### Custom Serde  <a href="#custom-serde" id="custom-serde"></a>

Custom serdes should be embedded in a new Lenses SQL processor Docker image.

To build a custom Docker image, create the following directory structure:

```bash
mkdir -p processor-docker/serde
```

Copy your serde jar files under processor-docker/serde.

Create `Dockerfile` containing:

```dockerfile
FROM lensesioextra/sql-processor:4.2

ADD serde /opt/serde
ENV LENSES_SQL_RUNNERS_SERDE_CLASSPATH_OPTS=/opt/serde
```

Build the Docker.

```bash
cd processor-docker
docker build -t example/lsql-processor .
```

Once the image is deployed in your registry, please set Lenses to use it (lenses.conf):

```bash
lenses.kubernetes.processor.image.name = "your/image-name"
lenses.kubernetes.processor.image.tag = "your-tag"
```

{% hint style="warning" %}
**Keep in mind**

**Don't use the `LPFP_` prefix.**

Internally, Lenses prefixes all its properties with `LPFP_`.

Avoid passing custom environment variables starting with `LPFP_` as it may cause the processors to fail.
{% endhint %}