I'm thinking of creating a PR for this feature but wanted to discuss it a bit since it will require config, code, scheduled task and documentation.
My thought for config is (I'm using CINSARMY as an example but it could be any txt file that is one entry per line, Cisco Talos, 3coresec, Spamhause, blocklist.de, etc etc)
```
firewall {
group {
external-list CINSARMY {
url https://cinsscore.com/list/ci-badguys.txt
interval 12h
}
external-list MANUALBANS {
url /config/user-data/ip-bans.txt
interval 10m
}
}
name OUTSIDE-IN {
default-action-drop
rule 1 {
action drop
source {
group {
external-list CINSARMY
}
}
}
}
}
```
it would be url of file, url would use curl to fetch the file on interval, file would just load the local file from /config/user-data/
both would be processed and put into 1 of 2 groups, ipv4, ipv6 since community lists like CINS can have a mix
It would create nftable groups for both types using name just like address and network groups, so L_CINSARMY, L6_CINSARMY, etc etc. and you populate them on interval.
Then in the config we would need to allow using them in rules. In the VyOS config we just need one entry both variants which simplifies the config for the user and allows for shorter config display since lists like CINSARMY is 30k entries which make show unusable.
I've currently build a bash script that does most of this work, but I'd love to add this functionality directly into VyOS for speed (Python) and future proofing.
I'd like to open this for discussion to make sure the value is worth the effort before I start work.
- Is the feature something the community would like to see?
- Does proposed the config layout make sense?
- Does the backend flow makes sense?
- Would there be a good show firewall op to list the tables or search for an IP for debugging?
- Does nftables have a swap action like ipset has, if so I'd build the tables in tmp groups and swap and remove the tmp as this would be faster (Need to research this)
- Anything else I forgot to mention or consider for this task?
UPDATE 1:
- [x] Renamed item from `blocklist` to `list` as @n.fort pointed out, it could be used for more then blocklists
- [x] only ipv4 and ipv6 needs to be separated, nft don't need to separate address and network this most be relics from the ipset switch to nft
- [x] Determined this PR will be very similar to geoip, with the exception of each list will require it's on cronjob based on interval, reusing the `system task-scheduler` code and adding one param for the output file should be able to handle this very easily. We would only reuse the backend codes as to not bloat the config, thus tasks will not show under `system task-scheduler` in the config.
UPDATE 2:
- [x] in the PR it was asked to move the task-scheduler logic into the python vyos package, this is now complete and part of the PR
- [x] in the PR I was asked to change the `lists` category to `external-list` which is a WIP, was hoping more would weigh in on this change before I executed it, but it's all I have left now