Export/Backup T-Pot Data

[encient]

I've seen several discussion posts about importing and exporting or backing up the T-Pot data, but there is no full write-up or steps on it.

As mentioned in T-Pot README, all log files are stored in ~/tpotce/data. However, according to this issue, if you import the folder straight away (by replacing your ~/tpotce/data folder), Kibana will show fewer logs as compared to your original data. According to the author who replied to the mentioned issue post, it is because of the default mode of how logstash handles files (you can refer to the issue for more information).

Anyway, I've successfully imported the backup file to my local instance, so I'll be providing a full guide just in case anyone needs it.



Given the scenario, I want to export my public T-Pot and import it into my local T-Pot.

Step 1: Replace ~/tpotce/data of public T-Pot to local T-Pot

• Find a way to zip your ~/tpotce/data of your public T-Pot and transfer it to your local T-Pot
• Remove the contents of ~/tpotce/data of your local T-Pot, and replace it with the one you imported

Step 2: Extract the current logstash.conf file from your local T-Pot

• The steps can be found here, but you can also refer to the command below:

docker exec -it logstash bash
cd /etc/logstash/
cp logstash.conf /data/elk/logstash.conf
exit

• Then stop T-Pot by running systemctl stop tpot to prevent conflict as we are going to modify the file later

Step 3: Modify logstash.conf

• Now that we already have a copy of logstash.conf in /data/elk folder, we can modify this file according to our needs
• In input section, add these 3 lines for all file block (means all honeypots that you are using)

  start_position => "beginning"
  mode => "read"
  sincedb_path => "/dev/null"

• Here's an example, before editing:

file {
  path => ["/data/cowrie/log/cowrie.json"]
  codec => json
  type => "Cowrie"
}

• After editing:

file {
  path => ["/data/cowrie/log/cowrie.json"]
  codec => json
  type => "Cowrie"
  start_position => "beginning"
  mode => "read"
  sincedb_path => "/dev/null"
}

Step 4: Set correct permissions for your new logstash.conf

chmod 760 ~/tpotce/data/elk/logstash.conf
chown tpot:tpot ~/tpotce/data/elk/logstash.conf

Step 5: Modify ~/tpotce/docker-compose.yml to consume the new logstash.conf

• Add ${TPOT_DATA_PATH}/elk/logstash.conf:/etc/logstash/logstash.conf in volumes section
• Output:

logstash:
  container_name: logstash
  restart: always
  volumes:
   - ${TPOT_DATA_PATH}:/data
   - ${TPOT_DATA_PATH}/elk/logstash.conf:/etc/logstash/logstash.conf

Step 6: Start T-Pot service

• systemctl start tpot

I hope this helps!

[notapentester1]

Hey thank you for the article. Did you by any way find a way to reset the logs at a given point in time? Like say "right" now?