Rithik Bhandari

Scenario: "Cape Town": Borked Nginx

Rithik Bhandari's photo
Rithik Bhandari
·

2 min read

Problem - https://sadservers.com/newserver/capetown

From the description we know -

  1. nginx server running on port 80

  2. yeah, that's it

Step 1 : checking nginx status

systemctl status nginx

it's in failed status
Step 2 : gotta look at logs

cd /var/log/nginx
tail -n 30 error.log

(-n flag allows to view N number of lines from the end of the file)
tailing logs reveals 2 issues.

  1. unexpected ";" in /etc/nginx/sites-enabled/default

  2. 24: Too many open files

So first seems simple enough to address, its a syntax issue. Edit the file mentioned in error to remove ";"

Step 3 : Debugging (24: Too many open files)

2nd one is about the service not able to open enough files for it work. for reference - https://www.baeldung.com/linux/error-too-many-open-files

Let's first get the process ID for nginx

ps -aux | grep nginx

as per the blog, we can get limits max open files with this.

pid=31540
grep "Max open files" /proc/$pid/limits

Here, I faced a problem. PID of nginx kept changing as it was restarting on loop. I hadn't restarted nginx since fixing the syntax issue. A restart should fix this.

WIth a fixed PID, trying the same command gives me the limit.

Max open files 10 10 files

For systems using systemd, you can modify the limits in the service unit file. Locate the service unit file for your process, usually in /etc/systemd/system/ or /etc/systemd/system/<your-service-name>.service.

[Service]
LimitNOFILE=10

LimitNOFILE - Maximum number of file descriptors allowed to be opened simultaneously
i edited this from 10 to 1024. Then restarted nginx service.

sudo systemctl restart nginx
Warning: The unit file, source configuration file or drop-ins of nginx.service changed on disk. Run 'systemctl daemon-reload' to reload units.

It failed,
asked to 'systemctl daemon-reload'

After that, nginx reload work and voila. We have a working nginx now!

Some Extra reading to understand everything better -

  1. What are file descriptors? - https://stackoverflow.com/a/5256705

  2. https://www.tecmint.com/exploring-proc-file-system-in-linux/

SadServers