Educating the world

Our blog has over 10,000 readers a month

Apache Access forbidden with extended mac @ permissions

December 13th, 2015

I came across an interesting file protection problem with my environment. OSX, Firefox and tar are all involved but I can't tell which component is doing it. The problem manifested itself with Access forbidden messages from Apache.

My objective is to download a customer's website and recreate it on our development system. Simple done it a million times, but strangely I ran into all sorts of problems when I tried to do it on my Mac.

Here is what I did:

  1. Logged into web site's Cpanel with Firefox.
  2. Zipped up the public_html folder.
  3. Downloaded it.
  4. Created a folder under my web root.
  5. Unzipped the files into their new home.
  6. Use Firefox (again) to look at the mirrored site. Firebug was reporting that about half the CSS files were not found (404) and all the image files were Access forbidden (403).
  7. Checked the code and the paths and everything is as it should be. It should be working!!

Went into the folder with the broken images and did a listing to see what's what.

total 432
-rw-r--r--@ 1 mrn  admin  24612 May 10  2015 File01.jpg
-rw-r--r--@ 1 mrn  admin  26618 May 19  2015 File02.JPG
-rw-r--r--@ 1 mrn  admin  54750 Feb 23  2014 File03.JPG

Maybe my Access problems were related to the mysterious @ symbol. A bit of Googling says that the @ sign means the file has extended attributes on top of the standard read-write-execute over owner-group-other permissions that we know and love. You can list the extended attributes with ls -l@

total 432
-rw-r--r--@ 1 mrn  admin  24612 May 10  2015 File01.jpg
	com.apple.quarantine	   26 
-rw-r--r--@ 1 mrn  admin  26618 May 19  2015 File02.JPG
	com.apple.quarantine	   26 
-rw-r--r--@ 1 mrn  admin  54750 Feb 23  2014 File03.JPG
	com.apple.quarantine	   26 
 

We can see that all the files are marked with com.apple.quarantine a quick check and the original zip file was indeed extended:

myhost:images mrn$ ls -l@ ~/Downloads/ec.tgz 
-rw-r--r--@ 1 mrn  staff  2490100 Dec  2 22:38 /Users/mrn/Downloads/compressed-file.tgz
	com.apple.quarantine	     26 

This could be the cause of the problem so how do we remove it. It looks like the extended tags have been applied to all the files in the archive, which kind of makes sense, but still it's a bit annoying as there weren't any warnings and it seems to present in an inconsistent way. So probably the best thing to do would be to remove the quarantine attribute from the original archive then uncompress it again. Let's give it a go using the clear switch on the extended attributes command:

myhost:images mrn$ ls -l@ ~/Downloads/ec.tgz 
-rw-r--r--@ 1 mrn  staff  2490100 Dec  2 22:38 /Users/mrn/Downloads/ec.tgz
	com.apple.quarantine	     26 
myhost:images mrn$ xattr -c /Users/mrn/Downloads/ec.tgz
myhost:images mrn$ ls -l@ ~/Downloads/ec.tgz 
-rw-r--r--  1 mrn  staff  2490100 Dec  2 22:38 /Users/mrn/Downloads/ec.tgz

So unzip and reload the page. Still didn't work. Over the next hour I played with copying the images to different folders until they loaded properly. I eventually figured out that everything under the main unzipped folder behaved strangely. Then it dawned on me that the only thing that could do this would be an .htaccess file and sure enough there it was. It contained the following line which matches URLs which end with an image extension and forbids them.

RewriteRule .*\.(jpg|jpeg|gif|png|bmp)$ - [F,NC]

This configuration is in place on the customer's live server. I renamed my mirrored .htaccess to _htaccess, reloaded the page and everything sprang into life. So their server must have been ignoring the .htaccess file, probably because it isn't an Apache web server so it doesn't understand what do do with it.

So the whole thing had nothing to do with permissions or extended attributes and was merely a straight forward under-sight: I should have checked for .htaccess files. It was an honest mistake as almost all web servers run Apache or something that understands .htaccess files! I have at least demonstrated how a system's administrator would go about solving a problem and we have learnt something along the way. Believe me, you get extremely good at tracking down weird problems when you run a large multi-user system and this one only took an hour to find!

Example log4j.properties file

October 1st, 2015

Configuring the Log4J properties file is one of those tasks which you have to spend a while trying to figure out but you invariably end up with the same thing. In fact most of the time something you cut and paste from the internet will do just fine. So here's mine!

This configuration outputs lines to a file and stdout.

# Root logger option
log4j.rootLogger=INFO, stdout, file

# Direct log messages to a log file
log4j.appender.file=org.apache.log4j.RollingFileAppender

log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.Target=System.out
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} %-5p %c{1}:%L - %m%n

log4j.appender.file.File=logs/my-app.0.0.log
log4j.appender.file.MaxFileSize=10MB
log4j.appender.file.MaxBackupIndex=10
log4j.appender.file.layout=org.apache.log4j.PatternLayout
log4j.appender.file.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} %-5p %c{1}:%L - %m%n

Example output:

2015-10-01 09:54:52 INFO ShortClassName:lineno - message

Pearls of wisdom from Bruce Lee

August 9th, 2015

Bruce Lee

A wise man can learn more from a foolish question than a fool can learn from a wise answer.

More sage words from Bruce on Brainy Quotes.

log4j.properties made simple

May 23rd, 2015

I wrote a huge article titled "Configuring log4j.properties" on, funnily enough, how to configure Log4J but sometimes it still isn't as good as great example:

log4j.rootLogger = INFO, X
log4j.logger.A.B.C=ALL, X
# or this for custom logger log4j.logger.A.B.C=A.B.C.MyLogger

#set the appender named X to be a console appender
log4j.appender.X=org.apache.log4j.ConsoleAppender

#set the layout for the appender X
log4j.appender.X.layout=org.apache.log4j.PatternLayout
log4j.appender.X.layout.conversionPattern=%m%n

#log4j.logger.A.B.C.appender= blar...

[ With the help of http://www.java2s.com/Code/Java/Language-Basics/Examplelog4jConfigurationFile.htm ]

Eclipse plugins

March 4th, 2015

I'm constantly trying new versions of Eclipse. I tend to be a fresh-install-reinstall-plugins kind of guy rather than a check-for-updates kind of guy. There always seems less that can go wrong when you reinstall everything from scratch each time, plus it's a chance to clean out all those temp folders. Of course there are a few drawbacks like having to keep tweaking your configuration settings but over all I prefer it this way.

The locations of plugins are the main things that I need to remember. Some versions of Eclipse come without the Market Place which is now itself a plugin! So I'm creating this blog to help me remember all the plugins I like to use.

Start Explorer - opens file manager or shell at package explorer level
http://basti1302.github.com/startexplorer/update/

PMD - a bit like lint
https://sourceforge.net/projects/pmd/files/pmd-eclipse/update-site/

ECL Emma - code coverage
http://update.eclemma.org/