[Patch] Direct (php-less) attachment downloads

General discussion of development ideas and the approaches taken in the 3.x branch of phpBB. The next feature release of phpBB 3 will be 3.3/Proteus.
Forum rules
Please do not post support questions regarding installing, updating, or upgrading phpBB 3.2.x. If you need support for phpBB 3.2.x please visit the 3.2.x Support Forum on phpbb.com.

If you have questions regarding writing extensions please post in Extension Writers Discussion to receive proper guidance from our staff and community.
intelx86
Registered User
Posts: 1
Joined: Thu Sep 04, 2003 12:03 am

[Patch] Direct (php-less) attachment downloads

Post by intelx86 » Fri Jul 23, 2010 6:31 am

Topic split off from [RFC|Accepted] Resuming for attachments / HTTP range support - bantu

I have already implemented direct downloads in my forum by inserting the following piece of code inside download/file.php
FIND

Code: Select all

	// Now the tricky part... let's dance
BEFORE, ADD

Code: Select all

	// Read and write for owner, read for everybody else
	chmod($filename, 0644);
	
	// Set expire time
	// size (MB) * 180s or 180s
	$exp = max(intval($size/1024/1024*180), 180);
	
	$dir = time().'_'.$exp.'_'.$user->data['session_id'].'_'.rand();
	if(!mkdir('./tmp/'.$dir)){
		echo 'Something went wrong creating a directory named "'. $dir .'"';
		return 0;
	}
	
	$real_filename = htmlspecialchars_decode($attachment['real_filename']);
	
	symlink('../../../files/'.$attachment['physical_filename'], './tmp/'.$dir.'/'.$real_filename) or die("Unable to create download link");


	// Close the db connection before sending the file
	$db->sql_close();
	
	$entities =     array('%25', '%5C', '%3F', '%23', '%5B', '%5D');
	$replacements = array( "%",  "\\",   "?",   "#",   "[",   "]" );

	header('Location: ./tmp/'.$dir.'/'.str_replace($replacements, $entities, $real_filename));
	file_gc();
Create a folder named tmp under phpbb_root/download/ and inside tmp a file named clean.sh with the following content:

Code: Select all

#! /bin/bash
time=`date +%s`
cd '/home/user/public_html/phpBB/download/tmp/'
#echo $time
for folder in $(ls -d */)
do
	#echo "$folder"
	IFS="_"
	name=( $folder )
	t=${name[0]}
	d=${name[1]}
	if [ "$time" -gt "$((t + d))" ]; then
		#echo "$folder"
		rm -rf "$folder"
	fi
done
Make the file clean.sh executable and set it as a cron job repeating every 5 minutes.

And last but not least a .htaccess file inside the tmp directory with the following content:

Code: Select all

Options -Indexes

<Files clean.sh>
	order deny,allow
	deny from all
</Files>

User avatar
Arman_Ajn
Registered User
Posts: 12
Joined: Fri May 21, 2010 6:43 pm
Contact:

Re: [RFC|Accepted] Resuming for attachments / HTTP range sup

Post by Arman_Ajn » Fri Jul 23, 2010 6:41 am

I'm sorry but i didn't get this part . will you explain it a bit ;)
Make the file clean.sh executable and set it as a cron job repeating every 5 minutes.

User avatar
DavidIQ
Customisations Team Leader
Customisations Team Leader
Posts: 1787
Joined: Thu Mar 02, 2006 4:29 pm
Location: Earth
Contact:

Re: [RFC|Accepted] Resuming for attachments / HTTP range sup

Post by DavidIQ » Fri Jul 23, 2010 6:44 am

That's great...now where's the Windows version? ;)
Almost your entire code is dependent on the server being Unix based...
Granted, it's a good step, but it's still only adressing the RFC on one server type.
Image

User avatar
bantu
3.0 Release Manager
3.0 Release Manager
Posts: 557
Joined: Thu Sep 07, 2006 11:22 am
Location: Karlsruhe, Germany
Contact:

Re: [RFC|Accepted] Resuming for attachments / HTTP range sup

Post by bantu » Fri Jul 23, 2010 8:17 am

intelx86 wrote:I have already implemented direct downloads in my forum by inserting the following piece of code inside download/file.php
Thanks for sharing your code. If I would have to implement direct downloads I would have done it in a similar fashion. However, as has been pointed out, there are a few issues that block such code going into the phpBB core:
  • phpBB 3.0.x right now never tries to create directories, because this can be an issue when safe_mode is on (if I remember correctly).
  • Your code assumes that the webserver reads and resolves symlinks, which doesn't have to be the case.
  • If someone knows the URL while it is valid, he/she can access the file, although it might be in a forum where he/she doesn't have access to.
The first two reasons should make this an optional feature only. I'm not sure whether we can detect whether the webserver resolves symlinks at all. The third point could be 'fixed' by delivering attachments that are not accessible to the guest user using the download/file.php script, all publically accessible downloads using direct download. This would still improve performance on big boards with many public forums, because PHP doesn't have to be involved in all downloads.
DavidIQ wrote:That's great...now where's the Windows version? ;)
Almost your entire code is dependent on the server being Unix based...
NTFS supports symlinks nowadays and PHP supports symlink() on Windows since PHP 5.3. And I'm pretty sure you can run bash scripts on Windows too. The bash script could probably also be easily turned into a phpBB cronjob, which also makes the .htaccess change unnecessary.

User avatar
DavidIQ
Customisations Team Leader
Customisations Team Leader
Posts: 1787
Joined: Thu Mar 02, 2006 4:29 pm
Location: Earth
Contact:

Re: [RFC|Accepted] Resuming for attachments / HTTP range sup

Post by DavidIQ » Fri Jul 23, 2010 8:21 am

bantu wrote:NTFS supports symlinks nowadays and PHP supports symlink() on Windows since PHP 5.3. And I'm pretty sure you can run bash scripts on Windows too. The bash script could probably also be easily turned into a phpBB cronjob, which also makes the .htaccess change unnecessary.
Yes I know you can do most of that if you tinker around with it but that's pretty much assuming the user has complete control over the server to do and install such things and, in most cases, they don't.
Image

User avatar
bantu
3.0 Release Manager
3.0 Release Manager
Posts: 557
Joined: Thu Sep 07, 2006 11:22 am
Location: Karlsruhe, Germany
Contact:

Re: [RFC|Accepted] Resuming for attachments / HTTP range sup

Post by bantu » Fri Jul 23, 2010 11:03 am

Resume support is also not always given in the solution proposed by intelx86. You can only resume the download within "180s * Mebibytes of Attachment".

User avatar
FeyFre
Registered User
Posts: 29
Joined: Wed Mar 17, 2010 9:49 pm

Re: [RFC|Accepted] Resuming for attachments / HTTP range sup

Post by FeyFre » Fri Jul 23, 2010 6:40 pm

bantu wrote:NTFS supports symlinks nowadays
Did I missed something? Since when? Since Vista? Sorry, neither Vista, nor 7, nor 2008 will be never suitable server platforms. Will this feature be available on 2000/XP/2003? No. Since XP/2003 will live until up to 2014 - symlinks will be not available for most users long time. So usage symlimks is bad way to implementation subjected feature.

User avatar
bantu
3.0 Release Manager
3.0 Release Manager
Posts: 557
Joined: Thu Sep 07, 2006 11:22 am
Location: Karlsruhe, Germany
Contact:

Re: [RFC|Accepted] Resuming for attachments / HTTP range sup

Post by bantu » Fri Jul 23, 2010 6:44 pm

FeyFre wrote:
bantu wrote:NTFS supports symlinks nowadays
Did I missed something? Since when?
http://en.wikipedia.org/wiki/NTFS_symbolic_link

User avatar
DavidIQ
Customisations Team Leader
Customisations Team Leader
Posts: 1787
Joined: Thu Mar 02, 2006 4:29 pm
Location: Earth
Contact:

Re: [RFC|Accepted] Resuming for attachments / HTTP range sup

Post by DavidIQ » Fri Jul 23, 2010 6:55 pm

FeyFre wrote:Sorry, neither Vista, nor 7, nor 2008 will be never suitable server platforms.
*cough*

(assuming you actually meant 2008 as in Windows Server 2008 because as for Vista and 7 I'd have to agree)
Image

User avatar
FeyFre
Registered User
Posts: 29
Joined: Wed Mar 17, 2010 9:49 pm

Re: [RFC|Accepted] Resuming for attachments / HTTP range sup

Post by FeyFre » Sat Jul 24, 2010 8:37 pm

DavidIQ wrote:*cough*[/size]
Wki is not a source of objective data. I believe what I can touch and all web-servers I have touched was on Linux. That is my objective statistics.

Post Reply