Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

decouple redis from netbox #580

Open
mmguero opened this issue Feb 6, 2025 · 1 comment
Open

decouple redis from netbox #580

mmguero opened this issue Feb 6, 2025 · 1 comment
Assignees
Labels
docker Relating to docker and docker-compose as used by Malcolm netbox Related to Malcolm's use of NetBox
Milestone

Comments

@mmguero
Copy link
Collaborator

mmguero commented Feb 6, 2025

redis is used by NetBox but will shortly be used by strelka (#485) as well. I don't know that there's any tight coupling, but we should decouple redis from NetBox as far as the containers/environment variables/etc. goes. NetBox will still use it, but other things should be able to as well. This is more about code organization than functional changes, I think.

@mmguero mmguero added docker Relating to docker and docker-compose as used by Malcolm netbox Related to Malcolm's use of NetBox labels Feb 6, 2025
@mmguero mmguero added this to the v25.03.0 milestone Feb 6, 2025
@mmguero mmguero added this to Malcolm Feb 6, 2025
@mmguero mmguero moved this to Todo (develop) in Malcolm Feb 6, 2025
@mmguero mmguero modified the milestones: v25.03.0, v25.02.0 Feb 11, 2025
@mmguero mmguero self-assigned this Feb 11, 2025
@mmguero mmguero moved this from Todo (develop) to In Progress in Malcolm Feb 11, 2025
mmguero added a commit to mmguero-dev/Malcolm that referenced this issue Feb 11, 2025
@mmguero mmguero moved this from In Progress to Testing in Malcolm Feb 11, 2025
@mmguero
Copy link
Collaborator Author

mmguero commented Feb 11, 2025

Just need to double-check kubernetes functionality before closing.

mmguero added a commit to mmguero-dev/Malcolm that referenced this issue Feb 11, 2025
piercema added a commit to piercema/Malcolm that referenced this issue Feb 12, 2025
* Bump development for v25.01.0, also update copyright year

* bump netbox to v4.1.10, osd_transform to v2.18.0, and fluent-bit to v3.2.4

* for cisagov#354, work in progress for Malcolm directly accepting syslog

* for cisagov#354, work in progress for Malcolm directly accepting syslog; (dashboard)

* cisagov#543, add naviation pane to non-network dashboards

* bump jinja to 3.1.5

* Documentation for cisagov#354, syslog

* replace old filebeat input for syslog with tcp/udp input and syslog processor, for cisagov#354

* Documentation for cisagov#354, syslog

* install.py tweak for cisagov#354

* minor fix for for cisagov#354, set host.name correctly

* bump netbox to v4.11.1 and elasticsearch-dsl to v8.17.1

* start of cisagov#356, normalize winlogbeats

* WIP of cisagov#356, normalize winlogbeats

* WIP of cisagov#356, normalize winlogbeats

* WIP of cisagov#356, fix for a dashboard

* WIP of cisagov#356, normalize winlogbeats

* Work in progress for cisagov#541, making sure conn.log and known_services.log get the ICS protocols assigned to them corrrectly and tagged appropriately

* Work in progress for cisagov#541

* standardize ICS protocols in network.protocol field, so they all get tagged with 'ics' properly cisagov#541

* fix cisagov#533, allow keystores to be created on startup even in hedgehog mode

* forgot to add file for cisagov#356

* For cisagov#524, handle filenames with spaces in extracted_files_http_server.py

* work for cisagov#542, preserve custom field formatting for index pattern on update of index pattern

* work for cisagov#542, preserve custom field formatting for index pattern on update of index pattern

* bump yq to v4.45.1

* for cisagov#551, URL pivot links from dashboards to arkime

* for cisagov#551, URL pivot links from dashboards to arkime

* fix pivot from arkime to dashboards and vice-versa when using a traefik or other reverse proxy

* for cisagov#551, URL pivot links from dashboards to netbox

* for cisagov#551, URL pivot links from dashboards to netbox

* for cisagov#551, URL pivot links from netbox to arkime/dashboards

* start of cisagov#553, update zeek to v7.1.0

* cisagov#553, handle conn.log for zeek v7.1.0 and documentation update

* cisagov#553, handle postgresql.log

* cisagov#553, handle postgresql.log

* cisagov#553, added PostgreSQL dashboard

* for cisagov#551, URL pivot links in dashboards (ignore date/times)

* start of omron fins integration, cisagov#554

* wip omron fins integration, , cisagov#554

* arkime to v5.6.0

* bump logstash and filebeat to v8.17.0

* Fix nginx filebeat

* WIP omron fins integration, cisagov#554

* WIP omron fins integration, cisagov#554

* WIP omron fins integration, cisagov#554

* WIP omron fins integration, cisagov#554

* WIP omron fins integration, cisagov#554

* dashboards tweaks

* fix links for hh redirect download

* First pass at adding suricata socket optimization

* fix issue with nginx proxy

* Setting debug to false

* Fixing permissions for socket

* html formatting

* documentation for workaround for UFW software firewall for Malcolm ISO should automatically open ports for syslog cisagov#560)

* Bump for v25.02.0 development

* restore _config.yml

* fix version

* I don't think we need a seperate pod for the socket-based suricata, that's what the offline one does now anyway, right?

* restore some comments, black python style

* some tweaks for cisagov#457, pulled jjrush's branch into mine for some fixes

* some tweaks for cisagov#457

* allow suricata to spawn threads

* logging tweaks

* more flexible verbosity for suricata

* some tweaks for cisagov#457, try to wait until PCAP is finished processing before moving on

* First pass at adding suricata socket optimization

* Setting debug to false

* Fixing permissions for socket

* for cisagov#457, a few tweaks of the suricata pcap processing mode after reviewing @jjrush's code

* for cisagov#457, monitor suricata.log to know when PCAP is done processing

* for cisagov#457, monitor suricata.log to know when PCAP is done processing

* for cisagov#457, signal suricata rules to reload after update

* for cisagov#457, signal suricata rules to reload after update

* for cisagov#457, fix processing of other log types

* for cisagov#457, fix processing of other log types

* for cisagov#457, signal suricata rules to reload after update

* decrease verbosity for log

* fix logic for autoarkime/forcearkime

* some tweaks for cisagov#457, don't bother keeping track of when suricata is done with a PCAP file. just let filebeat handle it and pick up the resultant eve.json files directly

* Standardizing healthcheck scripts, updating docker-compose, updating kubernetes

* Adding livenessProbe to htadmin

* cisagov#457, handle multiple Suricata PCAP processing threads

* cisagov#574, clear screen after auth_setup when using Dialog mode

* add the related.user field to the 'nginx Access Logs' table

* bump fluent bit to v3.2.5

* fixed import of ECS templates

* handle ARKIME_PORT value formatted like a URL in the init of the API container

* cisagov#565, warn user about overwriting netbox passwords if they've already been set

* fix cisagov#559, ANSI color codes from croc displayed

* Exception in build triggers

* for cisagov#557, try building dirinit with arm runner

* cisagov#557, use arm-hosted runners for github build actions

* restore _config.yml

* a bit of cleanup for Dockefiles/health check scripts

* minor fixes for health checks

* Tweaks for health checks

* restore _config.yml

* Tweaks for health checks

* build tweaks for health scripts

* bump capa to v9.0.0

* workaround for issue blocking cisagov#475, integration of sigma rules

* improvements to workaround for issue blocking cisagov#475, integration of sigma rules

* improvements to workaround for issue blocking cisagov#475, integration of sigma rules

* for cisagov#475, automatically apply aliases via index templates

* for cisagov#475, starting on mappings for security analytics

* for cisagov#585, include corelight/zeek-long-connections plugin for long connections (wIP)

* for cisagov#585, include corelight/zeek-long-connections plugin for long connections (WIP)

* for cisagov#585, include corelight/zeek-long-connections plugin for long connections (WIP)

* for cisagov#585, include corelight/zeek-long-connections plugin for long connections (WIP)

* demo fix

* for cisagov#585, show long connection count on connections dashboard

* decouple redis from netbox (cisagov#580)

* one more minor change to cisagov#491, moved all container health scripts into one place to make it easier to keep track of them

* decouple redis from netbox (cisagov#580) and reorganized some of the other netbox password stuff

* updated fluent bit

* fix filebeat health

---------

Co-authored-by: Seth Grover <[email protected]>
Co-authored-by: Jason Rush <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
docker Relating to docker and docker-compose as used by Malcolm netbox Related to Malcolm's use of NetBox
Projects
Status: Testing
Development

No branches or pull requests

1 participant