Splunk Free: .htaccess Protection using Apache

You ever hear of Splunk? Splunk enables you to search and navigate all your logs and IT data in real time; logs, configurations, messages, traps and alerts, scripts and metrics. It’s an awesome tool to make it easier to monitor and watch your log files. Unfortunately, Splunk is expensive. How expensive? Try $5000 a year, for the cheapest license. Here’s the main problem; the free version of Splunk does not come with any user authentication, not even Admin authentication. This means that anyone can access your Admin area of Splunk, and can see any log files you have and can even set up new Splunks (log file watches). Let’s fix this!

I would have thought that a standard feature of Splunk would be at least Admin user authentication, but you can only get that with the professional version of Splunk. You have 30 days of the Professional version of Splunk, and then you must purchase the license after that. So, most individuals that just want to manage their log files remotely via the web, cannot afford and should not even purchase a Professional license, so the Free version is perfect. The lack of authentication kind of makes you turn your nose to Splunk, as this posses a security issue. Note one thing, when I say authentication, I mean username and password. You literally can access all admin features, including license information, just by going to the web address (which is usually a domain name on the default port 8000, e.g. http://domain.com:8000). This is totally rediculous. We can get around this by running a proxy within Apache and secure the subdomain (http://splunk.example.com/) with a .htaccess file.

Just A Few Things

The environment I’m running is Apache 2.x on a CentOS server and you must have root access to the server, as you will need to install Splunk and then make changes to the Apache server. Also, I presume that you already have a domain name and you are wanting to create a sub domain called splunk (splunk.domain.tld), that has some sort of user authentication.

Installing Splunk

Installing Splunk on a system using the RPM is very easy; almost too easy. First, you will want to download the current version of Splunk (3.1.3 at the time of writing). You can use compile it from the source if you would like, but this article will cover how to install Splunk using the RPM. After selecting the download you want (RPM), it will redirect you to a download page that will give your the wget URL for downloading Splunk; select and copy that full URL that it gives you. The link that I provide may be old, depending on when you read this post. Now, in your BASH prompt:

BASH

[root@server ~]# wget 'http://www.splunk.com/index.php/download_track?file=3.1.3/linux/splunk-3.1.3-28524-linux-2.6-x86_64.rpm&ac=&wget=true&name=wget&type=releases'

This will download Splunk into the current directory you are in. When the download has completed, you can start the install. The RPM install is the easiest, you just need to run one command:

BASH

[root@server ~]# rpm -i --force --prefix=/opt/splunk3.0/splunk splunk-path-to-rpm.rpm

You should see something close to the following:

BASH Output

----------------------------------------------------------------------
The Splunk Server has been installed in:
        /opt/splunk3.0/splunk/splunk

To start the Splunk Server, run the command:
        /opt/splunk3.0/splunk/splunk/bin/splunk start

To use Splunk's web interface, point your browser at:
        http://server:8000

Complete documentation is at http://www.splunk.com/r/docs
----------------------------------------------------------------------

Disabling SELinux

When you tell Splunk to start, it will create some files and directories and then check to see if SELinux is enforced. If you have SELinux enabled, then Splunk will not run correctly, and you will need to either disable SELinux, or configure SELinux to allow Splunk to run correctly (not covered in this article). You can temporarily stop SELinux, but unfortunately, Splunk looks at the selinux file, and checks to see if it is set to enforcing. If it is set to enforcing, then we will need to change this in the SELinux configuration file, which is located at /etc/sysconfig/selinux. Edit the selinux file and set the SELINUX=enforcing to SELINUX=disabled. Once you have done this, you will need to save the file and then stop SELinux in real-time, as changing the configuration file only tells SELinux to disable itself at boot-up. So, you will need to set run the command setenforce 0 to disable SELinux in real-time. If you do not do this, you can also reboot the system and it will take the new settings for SELinux.

Starting Splunk

As the documentation states, start the Splunk server:

BASH

[root@server ~]# /opt/splunk3.0/splunk/splunk/bin/splunk start

You will need to scroll down to the bottom of the license agreement and accept it to continue. It will run its init script and should start with no issues. After it starts, it will let you know that Splunk is running on port 8000 on the host name of the server; you can substitute the host name with the IP address of the server. In this case, the host name of the server is server, so we can access Splunk using http://server:8000. More than likely, you will actually have a domain name on a remote network/server, so you will access it by way of http://example.com:8000.

Just a few notes

Remember, if this is a new server, you might not have Apache started and your firewall might cause issues when trying to access Splunk on your server. Make sure Apache is started by running /etc/init.d/httpd status If it is not running it will say httpd is stopped. You will need to start it by running /etc/init.d/httpd start. It should start with no issues. Now, try connecting to your server, by opening a web browser using http://server:8000 (or whatever your hostname is, in this case we are using server). This should display your Splunk startup screen. This means that Splunk has successfully been installed and is ready to be used! Congrats.

Configuring Apache for Splunk .htaccess Protection

As stated before, Splunk doesn’t offer any user authentication by default, so we have to configure Apache to protect our Splunk logs so that no one else can view your log files, which can have some very valuable information in them. Let’s secure this drawback of the free version of Splunk and make it so that .htaccess can authorize a user login. In order to get this working, we have to configure Apache as a proxy server for the IP address and the server name.

Load the Apache Proxy Modules

Before we continue, you need to make sure that you have at least the mod_proxy, mod_proxy_http, and mod_proxy_connect Apache modules installed. Normally, these are installed and loaded by default, so you shouldn’t have to worry about this. To verify this, just type in httpd -M and make sure those modules are loaded.

Making the changes to httpd.conf

Now, it’s time to actually setup the proxy. What we are going for here is to redirect any requests for the IP address and server name (such as a subdomain of splunk.example.com) and redirect it to localhost on the port that Splunk is running on and serve the .htaccess from a localhost connection.

Editing /etc/httpd/conf/httpd.conf

<virtualhost x.x.x.x:80>
        ServerAdmin root@localhost
        ServerAlias splunk.example.com
        ProxyPass / http://127.0.0.1:8000/
        ProxyPassReverse / http://127.0.0.1:8000/
        ErrorLog logs/splunk.example.com-error_log
        CustomLog logs/splunk.example.com-access_log common
</virtualhost>

<proxy http://127.0.0.1:8000/*>
        Order deny,allow
        Deny from all
        Allow from all
        AuthName "splunk.example.com"
        AuthType Basic
        AuthUserFile /var/www/.htpasswd.users
        Require valid-user
</proxy>

Where x.x.x.x is your public IP Address.

Of course, you will need to configure this for you environment. Make sure you change the x.x.x.x to your public IP address and change example.com to you own domain. Additionally, if you would like to, you can change the splunk subdomain to whatever you would like to also. Just make sure you create and update you DNS information as needed. If you are going to have a splunk.example.com subdomain, make sure you have this configured in your DNS first before you do this (also allow for it to propagate). Also, make sure that you restart Apache, or else the new changes will not work:

Restarting Apache

/etc/init.d/httpd restart

Creating the .htaccess File

With the above configuration, you told Apache to use the .htpasswd.users file in the /var/www directory. You can follow my other article on how to configure .htaccess. If you plan on storing your .htaccess/.htpasswd files somewhere else, you will need to update your httpd.conf file to reflect the absolute location.

Closing Notes

Personally, I think the free version of Splunk should at least provide an admin user login, but that just isn’t something they are offering. Splunk, is very powerful and extremely helpful to see all your log files from one view. I don’t have alot of data that is written to my log files, however, the data that gets generated really helps to solve some issues. I guarantee that using Splunk will help you out greatly, especially if you have alot of custom logs that you are trying to manage manually.