I came across an interesting file protection problem with my environment. OSX, Firefox and tar are all involved but I can't tell which component is doing it. The problem manifested itself with Access forbidden messages from Apache.
My objective is to download a customer's website and recreate it on our development system. Simple done it a million times, but strangely I ran into all sorts of problems when I tried to do it on my Mac.
Here is what I did:
- Logged into web site's Cpanel with Firefox.
- Zipped up the public_html folder.
- Downloaded it.
- Created a folder under my web root.
- Unzipped the files into their new home.
- Use Firefox (again) to look at the mirrored site. Firebug was reporting that about half the CSS files were not found (404) and all the image files were Access forbidden (403).
- Checked the code and the paths and everything is as it should be. It should be working!!
Went into the folder with the broken images and did a listing to see what's what.
-rw-r--r--@ 1 mrn admin 24612 May 10 2015 File01.jpg
-rw-r--r--@ 1 mrn admin 26618 May 19 2015 File02.JPG
-rw-r--r--@ 1 mrn admin 54750 Feb 23 2014 File03.JPG
Maybe my access problems were related to the mysterious @ symbol. A bit of Googling says that the @ sign means the file has extended attributes on top of the standard read-write-execute over owner-group-other permissions that we know and love. You can list the extended attributes with ls -l@.
total 432
-rw-r--r--@ 1 mrn admin 24612 May 10 2015 File01.jpg
com.apple.quarantine 26
-rw-r--r--@ 1 mrn admin 26618 May 19 2015 File02.JPG
com.apple.quarantine 26
-rw-r--r--@ 1 mrn admin 54750 Feb 23 2014 File03.JPG
com.apple.quarantine 26
We can see that all the files are marked with com.apple.quarantine a quick check and the original zip file was indeed extended:
myhost:images mrn$ ls -l@ ~/Downloads/ec.tgz
-rw-r--r--@ 1 mrn staff 2490100 Dec 2 22:38 /Users/mrn/Downloads/compressed-file.tgz
com.apple.quarantine 26
This could be the cause of the problem so how do we remove it. It looks like the extended tags have been applied to all the files in the archive, which kind of makes sense, but still it's a bit annoying as there weren't any warnings and it seems to present in an inconsistent way. So probably the best thing to do would be to remove the quarantine attribute from the original archive then uncompress it again. Let's give it a go using the clear switch on the extended attributes command:
myhost:images mrn$ ls -l@ ~/Downloads/ec.tgz
-rw-r--r--@ 1 mrn staff 2490100 Dec 2 22:38 /Users/mrn/Downloads/ec.tgz
com.apple.quarantine 26
myhost:images mrn$ xattr -c /Users/mrn/Downloads/ec.tgz
myhost:images mrn$ ls -l@ ~/Downloads/ec.tgz
-rw-r--r-- 1 mrn staff 2490100 Dec 2 22:38 /Users/mrn/Downloads/ec.tgz
So unzip and reload the page. Still didn't work. Over the next hour I played with copying the images to different folders until they loaded properly. I eventually figured out that everything under the main unzipped folder behaved strangely. Then it dawned on me that the only thing that could do this would be an .htaccess file and sure enough there it was. It contained the following line which matches URLs which end with an image extension and forbids them.
RewriteRule .*\.(jpg|jpeg|gif|png|bmp)$ - [F,NC]
This configuration is in place on the customer's live server. I renamed my mirrored .htaccess to _htaccess, reloaded the page and everything sprang into life. So their server must have been ignoring the .htaccess file, probably because it isn't an Apache web server so it doesn't understand what do do with it.
So the whole thing had nothing to do with permissions or extended attributes and was merely a straight forward under-sight: I should have checked for .htaccess files. It was an honest mistake as almost all web servers run Apache or something that understands .htaccess files! I have at least demonstrated how a system's administrator would go about solving a problem and we have learnt something along the way. Believe me, you get extremely good at tracking down weird problems when you run a large multi-user system and this one only took an hour to find!
