R1Q2 - HTTP download failed: Operation was aborted by an application callback

[r1]

Hello,
I'm trying to track down the following HTTP download error that keeps occurring for some people on the whale server. I've looked at the r1q2/curl code but I'm not sure why it's failing (i.e. can't connect or resolve host/proxy). Perhaps there could be something misconfigured or changed with the HTTP server or its settings. I appreciate your help! 


server.cfg: (our http setting)
set sv_downloadserver "http://http.tastyspleen.net/~p1/quake2/"

-------------

q2logfile: (error message)
[2011-05-22 17:47] Fatal HTTP error: No error
[2011-05-22 17:47] HTTP download failed: Operation was aborted by an application callback

------------------

R1Q2:
static void CL_FinishHTTPDownload (void)                                                    //      r1q2\client\cl_http.c(788):
{
   msg = curl_multi_info_read (multi, &msgs_in_queue);
   result = msg->data.result;
      ...
   switch (result)
   {
             ...
      case CURLE_COULDNT_RESOLVE_HOST:
      case CURLE_COULDNT_CONNECT:
      case CURLE_COULDNT_RESOLVE_PROXY:
             ...
      Com_Printf ("Fatal HTTP error: %s\n", LOG_CLIENT|LOG_WARNING, curl_easy_strerror (result));       //      r1q2\client\cl_http.c(904):
             ...            
      Com_Printf ("HTTP download failed: %s\n", LOG_CLIENT|LOG_WARNING, curl_easy_strerror (result));    //       r1q2\client\cl_http.c(916):
   }
}

-----

CURL:
curl_easy_strerror(CURLcode error)
{
   switch (error)
   {
      case CURLE_OK:                   return "No error";                            //0    curl\lib\strerror.c(58):
             ...
      case CURLE_ABORTED_BY_CALLBACK:    return "Operation was aborted by an application callback";       //42    curl\lib\strerror.c(160):
   }
}

[X7]

A HTTP bug that has not been fixed..
http://www.r1ch.net/forum/index.php?topic=608.msg10869#msg10869

I asked R1ch for this in a bug report on HTTP download. If it doesn't exist on the HTTP server, the client can't fall back to normal download. Rich declined to modify the code to allow UDP fail-over. I hope you have better luck.

 

[r1]

Thanks for looking into this X7. I'm already aware of the other error you mentioned, but I don't think it's the same.
In our case all the files must exist because the HTTP server is exactly the same as the game server / FTP where we upload to.
In fact we set it up this way as a work around to the error you mentioned, so we wouldn't have duplicates or missing files, but as a result run into the current problem 

[quadz]

[2011-05-22 17:47] Fatal HTTP error: No error
[2011-05-22 17:47] HTTP download failed: Operation was aborted by an application callback

I recall encountering this error a year or two ago, and discussing the issue with r1ch; but I don't recall the specifics now.

When this error occurs in Q2, what is the corresponding information in your web server access.log and error.log ?

There were two main issues I encountered while getting r1q2 HTTP downloads to work reliably:

- Had to mix baseq2 into the mod directory (in the HTTP download area), because the HTTP downloader only checks the mod directory and doesn't fall back to baseq2.

- Case sensitivity. Various maps are inconsistent about referencing textures, envs, etc. with upper or lowercase paths and filenames.  To solve this, I keep everything lowercased in the HTTP download area, and use an apache URL rewrite line to force all download attempts to lowercase.


Only since then have I stopped seeing HTTP download errors.

[Zeppelin]

the map override generator is also down 

[X7]

the map override generator is also down 

server must be where Hurricane Irene Slamming East Coast 

[r1]

the map override generator is also down 

BTW, you can use Map Mixer to edit and generate overrides in far fewer steps, and without having to visit an online generator which could be subject to such interruptions.  Another cool thing is, you can also save the override and launch quake 2 to preview your changes seamlessly all with one click, and from one simple user friendly interface:



http://tastyspleen.net/quake/forums/index.php?topic=16667.0

[NIN-IceFox]

[2011-05-22 17:47] Fatal HTTP error: No error
[2011-05-22 17:47] HTTP download failed: Operation was aborted by an application callback

I recall encountering this error a year or two ago, and discussing the issue with r1ch; but I don't recall the specifics now.

When this error occurs in Q2, what is the corresponding information in your web server access.log and error.log ?

There were two main issues I encountered while getting r1q2 HTTP downloads to work reliably:

- Had to mix baseq2 into the mod directory (in the HTTP download area), because the HTTP downloader only checks the mod directory and doesn't fall back to baseq2.

- Case sensitivity. Various maps are inconsistent about referencing textures, envs, etc. with upper or lowercase paths and filenames.  To solve this, I keep everything lowercased in the HTTP download area, and use an apache URL rewrite line to force all download attempts to lowercase.


Only since then have I stopped seeing HTTP download errors.

From my ongoing investigation I'm not sure if I've found any info in access.log and error.log that has matched the times when I've personally run into "Fatal HTTP error: No error" when connecting to our server. But here are the parts nearest to when I had a test,

access.log:

123.125.71.97 - - [05/Sep/2011:13:38:26 -0700] "GET / HTTP/1.1" 200 1594
123.125.71.89 - - [05/Sep/2011:14:12:19 -0700] "GET /robots.txt HTTP/1.1" 404 377
123.125.71.28 - - [05/Sep/2011:16:26:29 -0700] "GET / HTTP/1.1" 200 1594
220.181.108.97 - - [05/Sep/2011:16:26:43 -0700] "GET / HTTP/1.1" 200 1594
123.125.71.33 - - [05/Sep/2011:17:25:45 -0700] "GET / HTTP/1.1" 200 1594
220.181.108.81 - - [05/Sep/2011:17:26:03 -0700] "GET / HTTP/1.1" 200 1594
180.76.5.11 - - [05/Sep/2011:18:39:09 -0700] "GET /robots.txt HTTP/1.1" 404 373
65.52.109.146 - - [05/Sep/2011:19:04:28 -0700] "GET /robots.txt HTTP/1.1" 404 373
65.52.109.146 - - [05/Sep/2011:19:05:29 -0700] "GET / HTTP/1.1" 304 -
180.76.5.159 - - [05/Sep/2011:19:58:01 -0700] "GET / HTTP/1.1" 304 -

error.log:

[Mon Sep 05 00:17:58 2011] [error] [client 95.108.155.252] File does not exist: /home/p1/public_html/robots.txt
[Mon Sep 05 02:08:57 2011] [error] [client 66.249.72.10] File does not exist: /home/p1/public_html/robots.txt
[Mon Sep 05 05:39:20 2011] [error] [client 119.63.196.52] File does not exist: /home/p1/public_html/robots.txt
[Mon Sep 05 06:46:54 2011] [error] [client 180.76.5.92] File does not exist: /home/p1/public_html/robots.txt
[Mon Sep 05 08:53:41 2011] [error] [client 123.125.71.82] File does not exist: /home/p1/public_html/robots.txt
[Mon Sep 05 12:54:04 2011] [error] [client 119.63.196.56] File does not exist: /home/p1/public_html/robots.txt
[Mon Sep 05 13:34:25 2011] [error] [client 216.59.240.253] File does not exist: /home/p1/public_html/favicon.ico
[Mon Sep 05 13:34:47 2011] [error] [client 216.59.240.253] File does not exist: /home/p1/public_html/favicon.ico
[Mon Sep 05 14:12:19 2011] [error] [client 123.125.71.89] File does not exist: /home/p1/public_html/robots.txt
[Mon Sep 05 18:39:09 2011] [error] [client 180.76.5.11] File does not exist: /home/p1/public_html/robots.txt
[Mon Sep 05 19:04:28 2011] [error] [client 65.52.109.146] File does not exist: /home/p1/public_html/robots.txt

Also even when downloading just mod resources I've seen no inconsistency with file name case as seen below. This was on a map that only had stock q2 resources with nothing to be dl'd and it was lowercase name. Also HTTP(weapons.filelist): 404 File Not Found [23 remaining files] sometimes appears before the http downloads begin too.

HTTP(sound/weapons/doom_ssg.wav): 18948 bytes, 130.31kB/sec [75 remaining files]
HTTP(sound/weapons/blast6.wav): 51068 bytes, 157.82kB/sec [74 remaining files]
HTTP(sound/weapons/hum3.wav): 147424 bytes, 368.21kB/sec [73 remaining files]
HTTP(sound/weapons/saberon.wav): 14472 bytes, 104.69kB/sec [72 remainingfiles]
HTTP(sound/weapons/strike.wav): 69450 bytes, 344.28kB/sec [71 remaining files]
HTTP(sound/weapons/hum.wav): 67612 bytes, 258.93kB/sec [70 remaining files]
HTTP(sound/weapons/hum2.wav): 58174 bytes, 244.87kB/sec [69 remaining files]
HTTP(sound/weapons/sawidle.wav): 7545 bytes, 84.69kB/sec [68 remaining files]
Fatal HTTP error: No error
Downloading sound/ctf/tech1.wav
Downloading sound/misc/peep.wav
Downloading sound/ctf/tech2.wav
Downloading sound/misc/death1.wav
Downloading sound/misc/death3.wav
Downloading sound/misc/tele.wav
Downloading sound/misc/death4.wav
Downloading sound/misc/death5.wav

it seems like it times out or something at a specific points. Quite often lately I've even gotten "Fatal HTTP error: No error" upon connecting without any files to be downloaded and it's the last entry I see in my console when my R1Q2 client locks up randomly upon connecting, which I can seem to get past by disabling all downloading in my player setup, cause I already had all needed files.

Now, the only extra anomaly I pulled from the server qconsole log itself upon my connecting at any time is this,

[2011-09-05 23:28] NIN-KitsuneButt connected
[2011-09-05 23:28] NET_GetPacket: Connection refused from 65.98.84.18:27930
[2011-09-05 23:28] NIN-KitsuneButt[**.**.***.***:*****]: protocol 35: "R1Q2 b7904 x86 Oct 25 2008 Win32 RELEASE"

And it turns out this is happening for all other players as far as I can tell from my preliminary check of the log history. The IP address appears to be some sort of hosting service which I am not familiar with. I really don't know much about this stuff. So, beyond that I have no idea what else may be related to our problems.

[NIN-IceFox]

90% certain I ruled out that last part of my previous post about refused connection, seems it may be unrelated and is no longer present as far as I can tell.

Anyway, we are gearing up for a major update to WOD-X and due to the large and continuous flow of dynamic content we will be putting out, HTTP downloads are going to be a must as relying on UDP could be a showstopper. Countless time has been spent to no avail trying to figure out what is causing this, we have also tried setting up the El Paso server's HTTP source to whale's and that also gives the same result so I'm pretty sure the HTTP location being the same as the server isn't likely to be the cause.

- Had to mix baseq2 into the mod directory (in the HTTP download area), because the HTTP downloader only checks the mod directory and doesn't fall back to baseq2.

We do have all the .paks present in the weapons folder, far as I know that is all that is required for a server, right? Could you elaborate on that?

Another thing I do not fully understood is this



What role do these things I believe are called symbolic links do, and is there a chance they could be a factor in all this? There are lots of these things throughout the server directories and can't seem to get any info from them at all.

Now, I've also been getting a little extra error message now ever since I updated my client to b8012

Bad HTTP response code 403 for maps/marics44.bsp, aborting HTTP downloading.
Fatal HTTP error: No error
HTTP(weapons.filelist): 404 File Not Found [0 remaining files]
HTTP(weapons/maps/marics44.filelist): 404 File Not Found [0 remaining files]
HTTP download failed: Operation was aborted by an application callback
Downloading maps/marics44.bsp

So if anyone has a clue what the problem is we'd be very grateful for the assistance! It's the only major obstacle we have left for carrying out our plans on the server.

[quadz]

- Had to mix baseq2 into the mod directory (in the HTTP download area), because the HTTP downloader only checks the mod directory and doesn't fall back to baseq2.

We do have all the .paks present in the weapons folder, far as I know that is all that is required for a server, right? Could you elaborate on that?

For example, a server like railz runs from the "rails" directory, however the map list mixes maps both from "rails/maps" and from "baseq2/maps".

Unlike the UDP download, the HTTP download would simply abort if it couldn't find the map in "rails/maps".  It wouldn't try again in "baseq2/maps".

So, I wrote a script to underlay the baseq2 files / overlay the rails files into a single download location.

If you have 100% of the files you need in the weapons folder, this shouldn't be an issue for you.

What role do these things I believe are called symbolic links do, and is there a chance they could be a factor in all this? There are lots of these things throughout the server directories and can't seem to get any info from them at all.

The symlinks are used to save hard drive space.

We have a master Q2 repository on the server, and so there's no reason to duplicate files in your own quake2 directory, if they're the exact same file that's in our master repository.

It's possible the symlinks could cause a download problem, depending on the webserver configuration.  They are not inherently a problem though, my entire HTTP download site is 100% symlinks.  (I.e. http://q2http.tastyspleen.net/ )

There's a way to test this on your site.  Is maps/marics44.bsp a symlink?  If you use a regular web browser, can you download that file from your site?


Regards,

quadz

[NIN-IceFox]

The symlinks are used to save hard drive space.

We have a master Q2 repository on the server, and so there's no reason to duplicate files in your own quake2 directory, if they're the exact same file that's in our master repository.

It's possible the symlinks could cause a download problem, depending on the webserver configuration.  They are not inherently a problem though, my entire HTTP download site is 100% symlinks.  (I.e. http://q2http.tastyspleen.net/ )

There's a way to test this on your site.  Is maps/marics44.bsp a symlink?  If you use a regular web browser, can you download that file from your site?


Regards,

quadz

Yes actually it is a symlink on the whale database after looking just now, been checking some maps through http.tastyspleen.net/~p1/quake2/weapons/maps/ and it seems I can dl any map or resource there that is not a symlink or otherwise I get this in my firefox

Forbidden

You don't have permission to access /~p1/quake2/weapons/maps/leray32.bsp on this server.

Apache/2.2.9 (Debian) mod_python/3.3.1 Python/2.5.2 mod_ssl/2.2.9 OpenSSL/0.9.8g mod_perl/2.0.4 Perl/v5.10.0 Server at http.tastyspleen.net Port 80

Interesting, looks like this could possibly be the issue and some reason only UDP can grab those symlink files, at least usually that is. Matter of fact I'm pretty sure I got it once by http when I had all my dl's on and the http was unable to grab other related files first before it tried to get the map and then it got it. But not when I had all other downloads except maps and textures set to off in my player setup. So I gotta look at this a little more and see if I can come back with some more conclusions!

[|iR|Focalor]

I see the word "aborted" in the subject line, and I don't think it's too soon before Thanksgiving to say thank you to Quadz for installing the wheelchair ramp on the short bus just so Tubby could ride it.

[NIN-IceFox]

Seems there can be almost no doubt about it now, I have examined the WOD-X exclusive resources (models and their textures) and found not one symlink in them so I tried allowing only models to download and sure enough, no errors! Next thing I tried on whale is having both model and map downloads allowed and I picked maps that had no symlinked resources and they all downloaded with HTTP no trouble at all and no errors!

So the only question that remains is how this issue is to be addressed, is there some sort of script you can run or something to fix whatever may be wrong with them? If you could check them out real quick that'd be great, Haven't been able to get any info from them still. They've seemed pretty much untouchable.

[The Happy Friar]

Ok, time to necro this thread!  I have the same (simular?) issue. Here's my error:

Code: [Select]

Connecting to q2server.fuzzylogicinc.com...
108.178.55.70:27910: challenge
108.178.55.70:27910: client_connect
HTTP downloading enabled, URL: http://www.sterlingshield.net/home/steve/quake2/
couldn't exec r1gl.cfg




KICK THE P.A. by headshot
HTTP(holywars.filelist): 404 File Not Found [4 remaining files]
Oversized 404 body received (1546 bytes), aborting HTTP downloading.
Bad HTTP response code 404 for holywars.filelist, aborting HTTP downloading.
Fatal HTTP error: Maximum file size exceeded
HTTP(holywars/maps/hw1.filelist): 404 File Not Found [0 remaining files]
Oversized 404 body received (1546 bytes), aborting HTTP downloading.
Bad HTTP response code 404 for holywars/maps/hw1.filelist, aborting HTTP downloading.
Fatal HTTP error: Maximum file size exceeded
HTTP download failed: Operation was aborted by an application callback
HTTP download failed: Operation was aborted by an application callback
Downloading maps/hw1.bsp
]condump test
It doesn't download the map, q2 just sits there.
I have the max download size in my config: sv_max_download_size 10000000
I unpacked the holywars pak file to my server so it's content could be downloaded.  The structure on my server is:
http://www.sterlingshield.net/home/steve/quake2/holywars/maps & /models, etc.

Looks to be all lowercase.  Some times it downloads a model or to, some times not.

Any ideas?

[fdrjk]

Well, I'm getting a 404 error for that site, so you should probably work on that.