Home » release
Category Archives: release
We currently recommend a “side by side” or “replacement upgrade” for users moving from CHN 1.8 to 1.9, however we recognize that is not possible for every user of CHN. In this post, I’ll dive into the main steps that a user would need to take to upgrade from version 1.8 to 1.9.
I’ll presume that folks are using the chn-quickstart script to bootstrap their installs. For folks with custom path locations, etc., we’ll leave the translation to you. 🙂
Remove unsupported honeypots
Unfortunately, not all of the honeypots supported in 1.8 will work in 1.9, where we removed all Python 2 use. These include: amun, glastopf and wordpot. You can simply shut down those containers, or re-purpose those hosts later for supported honeypot types.
Upgrade the server
A note on those using
systemd to manage their CHN-Server instance: you’ll need to swap in the appropriate
systemctl commands instead of
docker-compose up/down commands below. You should also update your systemd service definition files to look more like the current best practice. Specifically ensure that your docker-compose commands do not include the
-v option, which removes named volumes. This is most important on honeypots, where registration data will be kept in named volumes, rather than volume mounts, going forward.
docker-compose down in
/opt/chnserver. If you’re already using chn-quickstart, you can
git checkout v1.9 to get the latest release version of the script. Run the latest version of
./guided_docker_compose.py and answer the questions asked. You’ll note there is a NEW question section: “Do you wish to enable intelligence feeds from a remote CIF instance?”.
This new section will enable the creation of a chn-intel-feeds container, which previously required manually adjusting the docker-compose.yml file. This container is used both to pull feeds from the CIF repository and re-publish them in a friendly “one-observable-per-line” format, but also is the mechanism by which partners can upload their own safelist items. These safelist items will apply to ANY feeds pulled by the partner, regardless of whether those feeds are pulled via chn-intel-feeds, CIF client, or raw API.
Once all the questions have been answered, a new docker-compose.yml file has been created, as well as all the necessary
env files. Note that we now use
env files rather than
sysconfig files. The primary difference (aside from name!) is how the files are loaded into the containers. The old
sysconfig files were volume mapped into the containers, and read as a file; the new
env files are loaded into the environmental variables of the container and read out of the running environment. We’ve found that this method is both more friendly for container orchestration environments and more reliable. Being read in via the docker-compose stanza
env_file, these new
env files do not require quotes around the variable values.
You can now explore the
./config/sysconfig directory and note any differences between the
sysconfig configurations, adjusting your
env files for any customizations needed (such as exporting hpfeeds-logs via syslog). If you’re submitting data to a CIF instance via hpfeeds-cif, you’ll want to adjust your
IGNORE_CIDR to include appropriate entries in the
hpfeeds-cif.env file. For those pulling feeds with
chn-intel-feeds, verify the feed specifications in
chn-intel-feeds.env meets your needs. And don’t forget about that new safelist functionality!
It’s been a long while since we released a new version; many of us were pulled in a different direction by March 2020. Rest assured that we have not been idle since the 1.8 release! We’re happy to announce version 1.9 of CommunityHoneyNetwork, which brings security updates as well as some new features for partners participating in the STINGAR community.
There was a lot of backend work which will be mostly transparent to the every day user, but was important for security and roadmap reasons:
- Removal of Python 2 code and dependencies and ansible build process (major re-factoring!)
- Removal of Amun, Wordpot, and Glastopf honeypots (due to lack of Python 3 support)
- Updated Cowrie, RDPhoney, UHP, Conpot, and Dionaea containers
- Improved logging (username/password logs from Dionaea, hpfeeds-logger support for syslog and file logging in single container, etc.)
- Per-partner safelisting uploads via chn-intel-feeds
- More k8s friendly honeypot containers (almost there!)
By far, the most work went into updating the CHN-Server and honeypot containers to ensure they’re using Python 3 (support for Python 2 ended in January). This also required us to update the backend data exchange code (hpfeeds -> hpfeeds3) and touch just about every part of the system. There are countless bug and performance fixes rolled into this release so hopefully the loss of the older honeypot types won’t be too painful.
We’re always interested in supporting the honeypot needs of the community, so if there’s a honeypot you’d like to see in the project, please file an issue on Github to let us know of your interest! Honeypots that support hpfeeds(3) or minimally have a plugin-based logging system are far easier to integrate, and of course we’re happy to take PR’s! 🙂
Users of the dionaea honeypot will see increased central logging due to some bug fixes and capture of logs with username and passwords. The hpfeeds-logger container now supports writing logs files and exporting via syslog in a single instance, reducing the need to run multiple containers.
We’ve deprecated our own Mongo and Redis containers in favor of upstream containers; we found that we currently have no need to customize these containers, so we’ re defaulting to upstream versions instead. Speaking of Redis, we’ve removed the need for Redis containers in CHN-Server (though users of hpfeeds-cif and hpfeeds-bhr will still need Redis for caching purposes). We’ve also removed Ansible as part of the container build process; we found that the ansible-based build process made our development processes very slow, and our experiences with the STINGAR customer base indicated that the ability to do local installs of honeypots/servers (v/s using container images) was never exercised.
More emphasis has been put on use of the chn-quickstart script for 1.9 deployments, and that repo is now using release tags. Please be sure to use a tagged release when starting up a new CHN instance to ensure you don’t catch us development and get unexpected results! 🙂
Unfortunately with all the major shifts in technology, the upgrade process from 1.8 to 1.9 will not be as simple as changing which tag your docker-compose.yml files are pointing to. We highly recommend spinning up a new server instance using the chn-quickstart script and re-deploying honeypots from the new console. For those wishing to retain log data, we’ve always recommended using the hpfeeds-logger container to write out files that are ingested elsewhere, so for most users throwing away the server instance and re-deploying the honeypots is a painless process. For those that choose to upgrade in place,
we’ll write up a follow up blog post on the process, but expect you’ll be fiddling with a lot of formatting and docker-compose format changes, which will (in our tests) take much longer than spinning up a new instance in another location.
Participants in the STINGAR data sharing project will be pleased at the inclusion of a mechanism to include their own safelist entries in the chn-intel-feeds container, which will safelist any incoming data against their own private safelists. Since many partners also desired safelisting against the Umbrella Top 10K domains (resolved to IP’s), we’ve added a process to automatically ingest those IP addresses on a daily basis. Partners can request this safelist be added to their partner key by sending an email to team-stingar[at]duke.edu. This feature should help those partners who are working to automatically ingest STINGAR feeds into protection devices feel more comfortable with the process, without having to “re-invent the safelisting wheel” as it were.
Finally, if you’re a CHN user, but not a STINGAR partner, we’d love to hear from you! We’d like to know how the project is working for you, and what you’d like to see in the future.We’re at a crossroads where we’re looking seriously at where we should invest our energy: do you need more TIP-like functionality? More honeypots or honeypot customization? New features for integrating into your existing SOAR platform, or automation in general? File an issue on Github or send us an email at team-stingar[at]duke.edu.
Stay safe out there folks!
We’re happy to announce the release of version 1.8 of CommunityHoneyNetwork. This release represents four months of feature development and bug fixes. A short list of things to look out for in this release include:
- ARM support for honeypots
- “Personality” support for Dionaea (and an upgrade to 0.8)
- Persistence of custom scripts via a volume mount
- CIDR exclusions for hpfeeds-cif (so you can exclude submitting your own IP addresses, etc.)
- A new container to feed a BHR instance directly from hpfeeds (which also includes CIDR exclusions)
- More flexible file rotation options for hpfeeds-logger (including specifying time unit, or “none” to allow for rotation outside the container)
- More robust tag support (and a suggested “key-value” format)
- Tons of small bug fixes, including removal of web interface items that aren’t in use
As always when upgrading, take a look at the documentation to ensure that there aren’t new variables that need to be added to your sysconfig files; those running hpfeeds-cif will want to check out the new IGNORE_CIDR variable, which replaces the previous “IGNORE_RFC1918” variable.
We’re very excited to announce the release of version 1.7 of CommunityHoneyNetwork! In addition to the usual bug fixes (updating dependencies, etc.), there are some great new features available in 1.7, such as:
- Honeypot tagging: Add a tag to your honeypot configuration that will show up in all logs. This allows some flexibility to add metadata about the collecting sensor directly in the log data.
- Custom ACME server support with Certbot: If you use a certificate provisioning service other than LetsEncrypt that supports the ACME protocol, you can have CHN-Server talk directly to it!
- Cowrie “Personalities”: Alter the SSH version, filesystem layout, output from commands, etc. using “personalities”. These are folders with bundles of cowrie configs that can be referenced in the sysconfig file to change the “look” of your cowrie honeypot, making it more difficult to identify.
- Dionaea bistreams log rotation: Given the proliferation of WannaCry on the open internet, we found that rotating the Dionaea bistreams logs to be critical in preventing near-daily filling up of disk space.
For an administrative perspective, we added features to delete events from the database when a sensor is deleted, removed a number of unsupported dependencies, and added lots of documentation on topics such as integrating honeypot services with systemd, and using the CIF client to pull feeds of data.
We’re also making a major change in the way we reference images in the docker-compose.yml and deployment scripts: rather than referencing ‘latest’, we’re specifying a specific version that matches with the server version. So for instance, with CHN-Server version 1.7, the deployment scripts will reference version 1.7 tagged images directly. This will allow for a number of useful things from our side, and ensure that users don’t inadvertently find themselves with mis-matched server and honeypot versions, or upgrading accidentally.
As with all upgrades in this project, we highly recommend that you take a fresh look at the documentation. As new features are added, there are often new options for sysconfig files that can impact the way your CHN instance functions. Once you’re comfortable with the changes, things should be as easy as a “docker pull” and “docker-compose down && docker-compose up -d” away! For the more risk-adverse/VM-rich, given the ease of deployment you can always spin up a new instance, fresh, and simply migrate old sensors to the new server via deployment scripts.
We hope you’ve enjoyed the holidays and are energized for a new year of security! We’re happy to announce a new version, 1.6, which has been tagged as latest. Those of you pulling images from DockerHub can enjoy the latest release by updating your server and honeypots with:
docker-compose stop && docker-compose pull && docker compose up -d
We’ve had this one running in our production environment for several weeks at this point, and briefly considered a Christmas week release, but decided it would be more of a gift to wait until after the holidays from an operational perspective. 🙂
This release includes some nice new features including:
- Tagging of honeypot type when submitting to CIF
- Initial (alpha) UHP support
- Automatic TLS certificate provisioning on CHN-Server via LetsEncrypt
- More logging format options with hpfeeds-logger (json!)
There’s also numerous bug fixes, including one for those upgrading from older versions that haven’t seen the pre-provisioned Deploy scripts show up yet. This fix ensures that those scripts are always populated when updating versions.
As always, please give these new versions a run in your environment and let us know how they perform. Visit our Github site to file issues or ask questions…