Autonomía digital y tecnológica

Código e ideas para una internet distribuida

Linkoteca. sysadmin

How to properly unban an IP address which Fail2Ban banned?

Imago voragine.net
• , ,

First, find the right jail to unban for. Use:

fail2ban-client status | grep "Jail list" | sed -E 's/^[^:]+:[ \t]+//' | sed 's/,//g'

to get the actual jail names. Then, use this to find the iptables rule name:

iptables -L -n

The rule name and jail name may not be the same but it should be clear which one is related to which.

Then, with Fail2Ban v0.8.8 and later:

fail2ban-client set unbanip

Where is the jail name, and is the IP address to unban.

With Fail2Ban before v0.8.8, the syntax is different:

fail2ban-client get actionunban

Grep Exclude: Patterns, Files, and Directories

Imago voragine.net
• , , ,

-v Invert match, show lines that do not match
-e pattern Specify a pattern (use multiple times to exclude several)
–exclude=»GLOB» Skip files matching the glob pattern
–include=»GLOB» Search only files matching the glob pattern
–exclude-dir=DIR Skip directories matching the name
-r Recursive search (skips symlinks found while descending)
-R Recursive search (follows all symlinks)

egrep command in Linux with examples

Imago voragine.net
• , , ,

‘egrep’ stands for «extended grep» and is a part of the grep family of commands used for text processing and pattern matching in Linux. It is preferred over basic grep when working with extended regular expressions, as it inherently supports meta-characters without requiring them to be escaped, thus simplifying complex searches. The ‘egrep’ command is also known for its speed and efficiency, making it a popular choice among system administrators and developers.

Running X window graphical application over ssh session

Imago voragine.net
• , ,

SSH the server has a feature called X11Forwarding. It specifies whether X11 forwarding is permitted or not. You need to set this parameter to yes. Once this is enabled, you will be able to run a graphical application on the server and get the application display (window) on your desktop.

For example, login on Linux/BSD system called server1-mydomain-com. Open the /etc/ssh/sshd_config file using text editor:

vi /etc/ssh/sshd_config

Find out parameter X11Forwarding and set it to yes:

X11Forwarding yes

How to ban an IP address in Fail2Ban

Imago voragine.net
• , ,

Identify the jail you want to apply the ban to:

sudo fail2ban-client status

Run the following command, replacing with the jail’s name (e.g., sshd) and with the IP you want to block:

sudo fail2ban-client set banip

Check if the IP is now listed under the banned IP list for that jail:

sudo fail2ban-client status

changeme. A default credential scanner

Imago voragine.net
• , , ,

changeme picks up where commercial scanners leave off. It focuses on detecting default and backdoor credentials and not necessarily common credentials. It’s default mode is to scan HTTP default credentials, but has support for other credentials.

changeme is designed to be simple to add new credentials without having to write any code or modules. changeme keeps credential data separate from code. All credentials are stored in yaml files so they can be both easily read by humans and processed by changeme. Credential files can be created by using the ./changeme.py --mkcred tool and answering a few questions.

Fixing the «Kernel Panic – not syncing: VFS: Unable to mount root fs on unknown-block(0,0)» Error After Upgrading Ubuntu

Imago voragine.net
• , ,

Regenerate initramfs for the New Kernel

To fix the issue, you need to regenerate the initramfs for the new kernel version. Run the following command in the terminal:

   sudo update-initramfs -u -k <version>

Replace <version> with the actual kernel version string for the kernel that you were unable to boot into. For example, it might look something like 4.15.0-36-generic.

You can find the kernel version by running uname -r if needed.

Update GRUB

Once the initramfs has been successfully generated, update the GRUB bootloader by running:

   sudo update-grub

This command ensures that GRUB recognizes the updated kernel and its corresponding initramfs.

Who is using chroot with php-fpm? Is it worth it?

Imago voragine.net
• , , , ,

I don’t use chroot, but the default setup for modern versions of FPM already compartmentalizes everything adequately for example, the private /tmp directory. I agree with others that chroot is an outdated way of doing things.

Also, I use SELinux…yet another way of achieving many of the same goals of chrooting. I’d highly recommend setting up SELinux if you are not already using it. If you’re concerned enough about security that you’d even think of chrooting php-fmp, you probably want to set up SELinux and have it on «Enforcing» (it’s useless on «Permissive» mode, that’s really only suitable for the configuration phase of test servers.) Not only will it provide security with PHP, but you get a whole bunch of other security benefits of it.

I have done some pretty sophisticated things with a web server under SELinux, requiring me to manually change a number of policies, and while I have had a few prolonged sessions of frustration, maybe 3-4 hours at a time of banging my head against the wall trying to get the permissions set up properly, it is totally worth it. It’s all up-front work, and once you learn how to do it it’s very easy.

Configure CGI executable Environment on Nginx

Imago voragine.net
• , , , , 
root@www:~# apt -y install fcgiwrap

root@www:~# vi /etc/nginx/fcgiwrap.conf
# create new
# for example, enable CGI under [/cgi-bin]

location /cgi-bin/ {
gzip off;
root /var/www;
fastcgi_pass unix:/var/run/fcgiwrap.socket;
include /etc/nginx/fastcgi_params;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
}

root@www:~# mkdir /var/www/cgi-bin

root@www:~# chmod 755 /var/www/cgi-bin
# add settings into [server] section of a site definition

root@www:~# vi /etc/nginx/sites-available/default

server {
.....
.....
include fcgiwrap.conf;
}

root@www:~# systemctl enable fcgiwrap

root@www:~# systemctl reload ngin

How to See All Devices on Your Network With nmap on Linux

Imago voragine.net
• , ,

nmap is a network mapping tool. It works by sending various network messages to the IP addresses in the range we’re going to provide it with it. It can deduce a lot about the device it is probing by judging and interpreting the type of responses it gets.

Let’s kick off a simple scan with nmap. We’re going to use the -sn (scan no port) option. This tells nmap to not probe the ports on the devices for now. It will do a lightweight, quick scan.

Even so, it can take a little time for nmap to run. Of course, the more devices you have on the network, the longer it will take. It does all of its probing and reconnaissance work first and then presents its findings once the first phase is complete. Don’t be surprised when nothing visible happens for a minute or so.

The IP address we’re going to use is the one we obtained using the ip command earlier, but the final number is set to zero. That is the first possible IPAddress on this network. The «/24» tells nmap to scan the entire range of this network. The parameter «192.168.4.0/24» translates as «start at IP address 192.168.4.0 and work right through all IP addresses up to and including 192.168.4.255».

Note we are using sudo.

sudo nmap -sn 192.168.4.0/24

How to enable HTTP/2 support in Apache

Imago voragine.net
• , ,

Enable HTTP/2 module
Apache’s HTTP/2 support comes from the mod_http2 module. Enable it from:

a2enmod http2
apachectl restart

If above commands do not work in your system (which is likely the case in CentOS/RHEL), use LoadModule directive in httpd configuration directory to enable http2 module.
Add HTTP/2 Support
We highly recommend you enable HTTPS support for your web site first. Most web browser simply do not support HTTP/2 over plain text. Besides, there are no excuses to not use HTTPS anymore. HTTP/2 can be enabled site-by-site basis. Locate your web site’s Apache virtual host configuration file, and add the following right after the opening tag:

Protocols h2 http/1.1

xdg and autostart in Linux X server regardless the desktop environment

Imago voragine.net
• , , , ,

There two main paths to look for entries to autostart:

/etc/xdg/autostart – called system-wide and most of the application will place files when they are installed.
[user’s home]/.config/autostart – user’s applications to start when the user logs in .

There is a security problem here, which is sometimes installing a package will place an autostart file there because the maintainer decided it is important but the package might be just a dependency and the next time the user logs in unwanted program might execute and open ports!

Xsession

Imago voragine.net
• , , ,

System-wide configuration of the Debian X session consists mainly of options inside the /etc/X11/Xsession.options file, and scripts inside the /etc/X11/Xsession.d directory. These scripts are all dotted in by a single /bin/sh shell, in the order determined by sorting their names. Administrators may edit the scripts, though caution is advised if you are not comfortable with shell programming.

Better font rendering in linux

Imago voragine.net
• , ,

If we’re not running a full blown desktop environment like GNOME, KDE or XFCE, the chances of getting good font rendering in a post Xorg installation after a default linux base (think of arch or voidlinux) install is zero. This guide serves as a list of todo items to get decent font rendering with these sort of installs.

Archiving a (WordPress) website with wget

Imago voragine.net
• , ,

I used wget, which is available on any linux-ish system (I ran it on the same Ubuntu server that hosts the sites).

wget –mirror -p –html-extension –convert-links -e robots=off -P . http://url-to-site

That command doesn’t throttle the requests, so it could cause problems if the server has high load. Here’s what that line does:

–mirror: turns on recursion etc… rather than just downloading the single file at the root of the URL, it’ll now suck down the entire site.
-p: download all prerequisites (supporting media etc…) rather than just the html
–html-extension: this adds .html after the downloaded filename, to make sure it plays nicely on whatever system you’re going to view the archive on
–convert-links: rewrite the URLs in the downloaded html files, to point to the downloaded files rather than to the live site. this makes it nice and portable, with everything living in a self-contained directory.
-e robots=off: executes the «robots off» command, telling wget to ignore any directive to ignore the site in question. This is strictly Not a Good Thing To Do, but if you own the site, this is OK. If you don’t own the site being archived, you should obey all robots.txt files or you’ll be a Very Bad Person.
-P .: set the download directory to something. I left it at the default «.» (which means «here») but this is where you could pass in a directory path to tell wget to save the archived site. Handy, if you’re doing this on a regular basis (say, as a cron job or something…)
http://url-to-site: this is the full URL of the site to download. You’ll likely want to change this.

Web ARChive

Imago voragine.net
• , , ,

The Web ARChive (WARC) archive format specifies a method for combining multiple digital resources into an aggregate archive file together with related information. The WARC format is a revision of the Internet Archive’s ARC_IA File Format[4] that has traditionally been used to store «web crawls» as sequences of content blocks harvested from the World Wide Web. The WARC format generalizes the older format to better support the harvesting, access, and exchange needs of archiving organizations. Besides the primary content currently recorded, the revision accommodates related secondary content, such as assigned metadata, abbreviated duplicate detection events, and later-date transformations.

Wget with WARC output

Imago voragine.net
• , , , ,

From the discussion about Working with ARCHIVE.ORG, we learn that it is important to save not just files but also HTTP headers.

To download a file and save the request and response data to a WARC file, run this:

wget «http://www.archiveteam.org/» –warc-file=»at»

This will download the file to index.html, but it will also create a file at-00000.warc.gz. This is a gzipped WARC file that contains the request and response headers (of the initial redirect and of the Wiki homepage) and the html data.

If you want to have an uncompressed WARC file, use the –no-warc-compression option:

wget «http://www.archiveteam.org/» –warc-file=»at» –no-warc-compression

Archiving Websites with Wget

Imago voragine.net
• , , ,

When IA first started doing their thing, they came across a problem: how do you actually save all of the information related to a website as it existed at a point in time? IA wanted to capture it all, including headers, images, stylesheets, etc.

After a lot of revision the smart folks there built a specification for a file format named WARC, for Web ARCive. The details aren’t super important, but the gist is that it will preserve everything, including headers, in a verifiable, indexed, checksumed format.