UFW software firewall for Malcolm ISO should automatically open ports for syslog #560
Assignees
Labels
Milestone
Comments
mmguero
added
bug
mmguero
added a commit
to mmguero-dev/Malcolm
that referenced
this issue
Jan 17, 2025
…O should automatically open ports for syslog cisagov#560)
mmguero
added a commit
to idaholab/Malcolm
that referenced
this issue
Jan 17, 2025
…O should automatically open ports for syslog cisagov#560)
|
I would argue you don't want to add any ports to default firewall rules. The less that occurs, the more secure the Malcolm deployment is, and as you said, the user can specify the port. I think it's a good approach to allow the "Malcolm" account to run a script to configure UFW software firewall. |
piercema
added a commit
to piercema/Malcolm
that referenced
this issue
Feb 12, 2025
* Bump development for v25.01.0, also update copyright year * bump netbox to v4.1.10, osd_transform to v2.18.0, and fluent-bit to v3.2.4 * for cisagov#354, work in progress for Malcolm directly accepting syslog * for cisagov#354, work in progress for Malcolm directly accepting syslog; (dashboard) * cisagov#543, add naviation pane to non-network dashboards * bump jinja to 3.1.5 * Documentation for cisagov#354, syslog * replace old filebeat input for syslog with tcp/udp input and syslog processor, for cisagov#354 * Documentation for cisagov#354, syslog * install.py tweak for cisagov#354 * minor fix for for cisagov#354, set host.name correctly * bump netbox to v4.11.1 and elasticsearch-dsl to v8.17.1 * start of cisagov#356, normalize winlogbeats * WIP of cisagov#356, normalize winlogbeats * WIP of cisagov#356, normalize winlogbeats * WIP of cisagov#356, fix for a dashboard * WIP of cisagov#356, normalize winlogbeats * Work in progress for cisagov#541, making sure conn.log and known_services.log get the ICS protocols assigned to them corrrectly and tagged appropriately * Work in progress for cisagov#541 * standardize ICS protocols in network.protocol field, so they all get tagged with 'ics' properly cisagov#541 * fix cisagov#533, allow keystores to be created on startup even in hedgehog mode * forgot to add file for cisagov#356 * For cisagov#524, handle filenames with spaces in extracted_files_http_server.py * work for cisagov#542, preserve custom field formatting for index pattern on update of index pattern * work for cisagov#542, preserve custom field formatting for index pattern on update of index pattern * bump yq to v4.45.1 * for cisagov#551, URL pivot links from dashboards to arkime * for cisagov#551, URL pivot links from dashboards to arkime * fix pivot from arkime to dashboards and vice-versa when using a traefik or other reverse proxy * for cisagov#551, URL pivot links from dashboards to netbox * for cisagov#551, URL pivot links from dashboards to netbox * for cisagov#551, URL pivot links from netbox to arkime/dashboards * start of cisagov#553, update zeek to v7.1.0 * cisagov#553, handle conn.log for zeek v7.1.0 and documentation update * cisagov#553, handle postgresql.log * cisagov#553, handle postgresql.log * cisagov#553, added PostgreSQL dashboard * for cisagov#551, URL pivot links in dashboards (ignore date/times) * start of omron fins integration, cisagov#554 * wip omron fins integration, , cisagov#554 * arkime to v5.6.0 * bump logstash and filebeat to v8.17.0 * Fix nginx filebeat * WIP omron fins integration, cisagov#554 * WIP omron fins integration, cisagov#554 * WIP omron fins integration, cisagov#554 * WIP omron fins integration, cisagov#554 * WIP omron fins integration, cisagov#554 * dashboards tweaks * fix links for hh redirect download * First pass at adding suricata socket optimization * fix issue with nginx proxy * Setting debug to false * Fixing permissions for socket * html formatting * documentation for workaround for UFW software firewall for Malcolm ISO should automatically open ports for syslog cisagov#560) * Bump for v25.02.0 development * restore _config.yml * fix version * I don't think we need a seperate pod for the socket-based suricata, that's what the offline one does now anyway, right? * restore some comments, black python style * some tweaks for cisagov#457, pulled jjrush's branch into mine for some fixes * some tweaks for cisagov#457 * allow suricata to spawn threads * logging tweaks * more flexible verbosity for suricata * some tweaks for cisagov#457, try to wait until PCAP is finished processing before moving on * First pass at adding suricata socket optimization * Setting debug to false * Fixing permissions for socket * for cisagov#457, a few tweaks of the suricata pcap processing mode after reviewing @jjrush's code * for cisagov#457, monitor suricata.log to know when PCAP is done processing * for cisagov#457, monitor suricata.log to know when PCAP is done processing * for cisagov#457, signal suricata rules to reload after update * for cisagov#457, signal suricata rules to reload after update * for cisagov#457, fix processing of other log types * for cisagov#457, fix processing of other log types * for cisagov#457, signal suricata rules to reload after update * decrease verbosity for log * fix logic for autoarkime/forcearkime * some tweaks for cisagov#457, don't bother keeping track of when suricata is done with a PCAP file. just let filebeat handle it and pick up the resultant eve.json files directly * Standardizing healthcheck scripts, updating docker-compose, updating kubernetes * Adding livenessProbe to htadmin * cisagov#457, handle multiple Suricata PCAP processing threads * cisagov#574, clear screen after auth_setup when using Dialog mode * add the related.user field to the 'nginx Access Logs' table * bump fluent bit to v3.2.5 * fixed import of ECS templates * handle ARKIME_PORT value formatted like a URL in the init of the API container * cisagov#565, warn user about overwriting netbox passwords if they've already been set * fix cisagov#559, ANSI color codes from croc displayed * Exception in build triggers * for cisagov#557, try building dirinit with arm runner * cisagov#557, use arm-hosted runners for github build actions * restore _config.yml * a bit of cleanup for Dockefiles/health check scripts * minor fixes for health checks * Tweaks for health checks * restore _config.yml * Tweaks for health checks * build tweaks for health scripts * bump capa to v9.0.0 * workaround for issue blocking cisagov#475, integration of sigma rules * improvements to workaround for issue blocking cisagov#475, integration of sigma rules * improvements to workaround for issue blocking cisagov#475, integration of sigma rules * for cisagov#475, automatically apply aliases via index templates * for cisagov#475, starting on mappings for security analytics * for cisagov#585, include corelight/zeek-long-connections plugin for long connections (wIP) * for cisagov#585, include corelight/zeek-long-connections plugin for long connections (WIP) * for cisagov#585, include corelight/zeek-long-connections plugin for long connections (WIP) * for cisagov#585, include corelight/zeek-long-connections plugin for long connections (WIP) * demo fix * for cisagov#585, show long connection count on connections dashboard * decouple redis from netbox (cisagov#580) * one more minor change to cisagov#491, moved all container health scripts into one place to make it easier to keep track of them * decouple redis from netbox (cisagov#580) and reorganized some of the other netbox password stuff * updated fluent bit * fix filebeat health --------- Co-authored-by: Seth Grover <[email protected]> Co-authored-by: Jason Rush <[email protected]>
piercema
added a commit
to piercema/Malcolm
that referenced
this issue
Feb 13, 2025
* Bump development for v25.01.0, also update copyright year * bump netbox to v4.1.10, osd_transform to v2.18.0, and fluent-bit to v3.2.4 * for cisagov#354, work in progress for Malcolm directly accepting syslog * for cisagov#354, work in progress for Malcolm directly accepting syslog; (dashboard) * cisagov#543, add naviation pane to non-network dashboards * bump jinja to 3.1.5 * Documentation for cisagov#354, syslog * replace old filebeat input for syslog with tcp/udp input and syslog processor, for cisagov#354 * Documentation for cisagov#354, syslog * install.py tweak for cisagov#354 * minor fix for for cisagov#354, set host.name correctly * bump netbox to v4.11.1 and elasticsearch-dsl to v8.17.1 * start of cisagov#356, normalize winlogbeats * WIP of cisagov#356, normalize winlogbeats * WIP of cisagov#356, normalize winlogbeats * WIP of cisagov#356, fix for a dashboard * WIP of cisagov#356, normalize winlogbeats * Work in progress for cisagov#541, making sure conn.log and known_services.log get the ICS protocols assigned to them corrrectly and tagged appropriately * Work in progress for cisagov#541 * standardize ICS protocols in network.protocol field, so they all get tagged with 'ics' properly cisagov#541 * fix cisagov#533, allow keystores to be created on startup even in hedgehog mode * forgot to add file for cisagov#356 * For cisagov#524, handle filenames with spaces in extracted_files_http_server.py * work for cisagov#542, preserve custom field formatting for index pattern on update of index pattern * work for cisagov#542, preserve custom field formatting for index pattern on update of index pattern * bump yq to v4.45.1 * for cisagov#551, URL pivot links from dashboards to arkime * for cisagov#551, URL pivot links from dashboards to arkime * fix pivot from arkime to dashboards and vice-versa when using a traefik or other reverse proxy * for cisagov#551, URL pivot links from dashboards to netbox * for cisagov#551, URL pivot links from dashboards to netbox * for cisagov#551, URL pivot links from netbox to arkime/dashboards * start of cisagov#553, update zeek to v7.1.0 * cisagov#553, handle conn.log for zeek v7.1.0 and documentation update * cisagov#553, handle postgresql.log * cisagov#553, handle postgresql.log * cisagov#553, added PostgreSQL dashboard * for cisagov#551, URL pivot links in dashboards (ignore date/times) * start of omron fins integration, cisagov#554 * wip omron fins integration, , cisagov#554 * arkime to v5.6.0 * bump logstash and filebeat to v8.17.0 * Fix nginx filebeat * WIP omron fins integration, cisagov#554 * WIP omron fins integration, cisagov#554 * WIP omron fins integration, cisagov#554 * WIP omron fins integration, cisagov#554 * WIP omron fins integration, cisagov#554 * dashboards tweaks * fix links for hh redirect download * First pass at adding suricata socket optimization * fix issue with nginx proxy * Setting debug to false * Fixing permissions for socket * html formatting * documentation for workaround for UFW software firewall for Malcolm ISO should automatically open ports for syslog cisagov#560) * Bump for v25.02.0 development * restore _config.yml * fix version * I don't think we need a seperate pod for the socket-based suricata, that's what the offline one does now anyway, right? * restore some comments, black python style * some tweaks for cisagov#457, pulled jjrush's branch into mine for some fixes * some tweaks for cisagov#457 * allow suricata to spawn threads * logging tweaks * more flexible verbosity for suricata * some tweaks for cisagov#457, try to wait until PCAP is finished processing before moving on * First pass at adding suricata socket optimization * Setting debug to false * Fixing permissions for socket * for cisagov#457, a few tweaks of the suricata pcap processing mode after reviewing @jjrush's code * for cisagov#457, monitor suricata.log to know when PCAP is done processing * for cisagov#457, monitor suricata.log to know when PCAP is done processing * for cisagov#457, signal suricata rules to reload after update * for cisagov#457, signal suricata rules to reload after update * for cisagov#457, fix processing of other log types * for cisagov#457, fix processing of other log types * for cisagov#457, signal suricata rules to reload after update * decrease verbosity for log * fix logic for autoarkime/forcearkime * some tweaks for cisagov#457, don't bother keeping track of when suricata is done with a PCAP file. just let filebeat handle it and pick up the resultant eve.json files directly * Standardizing healthcheck scripts, updating docker-compose, updating kubernetes * Adding livenessProbe to htadmin * cisagov#457, handle multiple Suricata PCAP processing threads * cisagov#574, clear screen after auth_setup when using Dialog mode * add the related.user field to the 'nginx Access Logs' table * bump fluent bit to v3.2.5 * fixed import of ECS templates * handle ARKIME_PORT value formatted like a URL in the init of the API container * cisagov#565, warn user about overwriting netbox passwords if they've already been set * fix cisagov#559, ANSI color codes from croc displayed * Exception in build triggers * for cisagov#557, try building dirinit with arm runner * cisagov#557, use arm-hosted runners for github build actions * restore _config.yml * a bit of cleanup for Dockefiles/health check scripts * minor fixes for health checks * Tweaks for health checks * restore _config.yml * Tweaks for health checks * build tweaks for health scripts * bump capa to v9.0.0 * workaround for issue blocking cisagov#475, integration of sigma rules * improvements to workaround for issue blocking cisagov#475, integration of sigma rules * improvements to workaround for issue blocking cisagov#475, integration of sigma rules * for cisagov#475, automatically apply aliases via index templates * for cisagov#475, starting on mappings for security analytics * for cisagov#585, include corelight/zeek-long-connections plugin for long connections (wIP) * for cisagov#585, include corelight/zeek-long-connections plugin for long connections (WIP) * for cisagov#585, include corelight/zeek-long-connections plugin for long connections (WIP) * for cisagov#585, include corelight/zeek-long-connections plugin for long connections (WIP) * demo fix * for cisagov#585, show long connection count on connections dashboard * decouple redis from netbox (cisagov#580) * one more minor change to cisagov#491, moved all container health scripts into one place to make it easier to keep track of them * decouple redis from netbox (cisagov#580) and reorganized some of the other netbox password stuff * updated fluent bit * fix filebeat health --------- Co-authored-by: Seth Grover <[email protected]> Co-authored-by: Jason Rush <[email protected]>
mmguero
added a commit
to mmguero-dev/Malcolm
that referenced
this issue
Feb 20, 2025
mmguero
added a commit
to mmguero-dev/Malcolm
that referenced
this issue
Feb 20, 2025
…that will maintain them (see cisagov#560)
mmguero
added a commit
to mmguero-dev/Malcolm
that referenced
this issue
Feb 24, 2025
mmguero
added a commit
to mmguero-dev/Malcolm
that referenced
this issue
Feb 24, 2025
piercema
added a commit
to piercema/Malcolm
that referenced
this issue
Mar 3, 2025
* After discovering opensearch-project/OpenSearch#17339, I have set `http.compression: false` and things are working correctly, so I'm putting v2.19.0 back in Revert "Revert "bump opensearch and dashboards to v2.19.0", as it seems to break arkime capture:" This reverts commit 54472cf. * working on normalizing winbeat names to match ECS * remove all xmlns fields * restructure templates so that things are objects rather than just flat * Added field to winlog * Added winlog.event_data.Context * improvements to winlog normalization * winlog normalization stuff * can't write to aliases dumbkoff * bump arkime to v5.6.1 * set github builds for staging * cisagov#555, document standards for supply chain and code provenance checking * cisagov#555, document standards for supply chain and code provenance checking * cisagov#555, document standards for supply chain and code provenance checking * cisagov#506, updates to documenation for Docker-based installation examples * not used yet, but added file for handling actions for environment variables after a malcolm upgrade * handling actions for environment variables after a malcolm upgrade * related to cisagov#580, in checkEnvFilesAndValues process env-var-actions to handle moved/deleted environment variables * cisagov#506, updates to documenation for Docker-based installation examples * Set redis_port and redis_cache_port * update documentation for cisagov#589, how to restart suricata to pick up new rules * update documentation for cisagov#589, how to restart suricata to pick up new rules * for cisagov#560, WIP for more robust port handling * limit default firewall rules on Malcolm ISO, now that we have script that will maintain them (see cisagov#560) * cisagov#560, working on dynamic UFW firewall port management for Malcolm ISO * cisagov#560, working on dynamic UFW firewall port management for Malcolm ISO * cisagov#547, lazy-load Faraday connections for netbox * don't fail healthcheck if pcap_capture is disabled * bump fluent bit to v3.2.7 * for cisagov#547, change caching parameters for netbox and added some (disabled by default) profiling code for netbox API calls * cisagov#598, create a little script for validating zeek policy * for cisagov#547; after understanding how logstash was cloning the netbox enrichment filter (and by extension the caches) I've reworked the caches so they're shared across batches (while remaining thread-safe). This results in about a 4x speedup. * added support for dnp3_control.clear_bit which I'm told will be merged into icsnpp-dnp3 within the next hour * minor reorganization * fix arkime desktop icon pointing to the wrong page * have docker make .xz version of raspi image * clear up space building raspi image * don't save .img after .img.xz is built * fix arkime icon link * Some updates to some of the demo/environment related scripts that aren't part of malcolm itself per-se, but used to set up malcolm in various environments * Some updates to some of the demo/environment related scripts that aren't part of malcolm itself per-se, but used to set up malcolm in various environments * Bump for v25.03.0 development --------- Co-authored-by: Seth Grover <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
#354 added support for syslog ingestion. However, in the ISO install of Malcolm the ports will not be open in the firewall, and require this workaround (depending on the ports specified):
We could just add
514/tcpand514/udpto the default firewall rules but the user is allowed to specify the port, so we can't just do that.The thing probably to do is to add an entry to
config/includes.chroot/etc/sudoers.d/to allow the user (technically users of thedockergroup or, maybe, thenetworkgroup) the run ufw as sudo without password, then adjust it when they set the ports either in theinstall.pyscript or upon startup.For now I will document that the user needs to run the UFW command manually.
The text was updated successfully, but these errors were encountered: