# Standalone SFTP pipeline (Benthos / Redpanda Connect)

Plain Benthos config that reads files from an SFTP server and prints them to stdout. Verified
against the public Rebex test server with the `benthos` binary shipped in `umh-core:v0.44.11`.

Config file: [`sftp.yaml`](sftp.yaml)

```yaml
input:
  sftp:
    address: test.rebex.net:22
    auto_replay_nacks: true
    credentials:
      username: demo
      password: password
      host_public_key: ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOdXzF+Jx/wvEBun5fxi8FQK30miLZFND0rxkYwNcYlE
    paths:
      - /pub/example/*
    scanner:
      to_the_end: {}
    watcher:
      enabled: false

buffer:
  none: {}

pipeline:
  processors:
    - bloblang: root = content()

output:
  stdout: {}
```

## Running it

Expected output: the log lines `Input type sftp is now active` / `Output type stdout is now active`,
then the file contents (binary noise for the Rebex sample PNGs). With `watcher.enabled: false` the
process exits once every matching file has been read.


## Field notes

- `paths` — list of globs, absolute on the remote server.
- `scanner.to_the_end: {}` — one message per file. Use `lines: {}` for one message per line,
  `csv: {}` for CSV rows.
- `watcher.enabled: false` — read the matching files once and finish. Set `true` (plus
  `watcher.poll_interval` and `watcher.cache`, which needs a `cache_resources` entry) to keep
  polling for new files.
- `output: stdout: {}` — testing only. Swap for `kafka`, `amqp_1`, `aws_s3`, etc. in real use.
- Key-based auth instead of a password: `credentials.private_key` (inline) or
  `credentials.private_key_file`, plus `credentials.private_key_pass` if encrypted.

## Getting `host_public_key`

The value is parsed with `ssh.ParseAuthorizedKey`, so it must be the bare authorized-keys form —
`<key-type> <base64>` — with no hostname prefix, no wildcard, and no comment needed.

1. Scan the server:

   ```bash
   ssh-keyscan -t ed25519 test.rebex.net
   ```

   Output (the leading `#` line is a banner on stderr):

   ```
   test.rebex.net ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOdXzF+Jx/wvEBun5fxi8FQK30miLZFND0rxkYwNcYlE
   ```

2. Drop the first field (the hostname). Paste the rest — `ssh-ed25519 AAAA…` — as `host_public_key`.

   One-liner that prints exactly what to paste:

   ```bash
   ssh-keyscan -t ed25519 test.rebex.net 2>/dev/null | grep -v '^#' | cut -d' ' -f2-
   ```

3. If the server has no ed25519 key, try `-t rsa` or `-t ecdsa`. Any key the server actually
   presents works; the plugin restricts the handshake to that key's algorithm.

4. Verify the fingerprint out-of-band with whoever runs the SFTP server before trusting it —
   `ssh-keyscan` output is only as trustworthy as the network you ran it on.

Alternative: put the same one-line content in a file and use `host_public_key_file: /path/to/key`.
Setting both fields at once is a lint error.

## Common failure

Wrong or placeholder host key value:

```
failed to init input <no label> path root.input: error parsing host public key: ssh: no key found; last parsing error for ignored line: illegal base64 data at input byte 0
```

Causes seen:

- known_hosts line pasted verbatim (hostname prefix included)
- wildcard / placeholder such as `test.rebex.net ssh-ed25519 *`
- key omitted entirely — the plugin then falls back to `~/.ssh/known_hosts`, which does not exist
  in a container image