apt-get update time out

[ieri]

Hi,

It worked fine during months and now (since 2 or 3 weeks), sudo apt-get update ends with a time out:

  Hit http://www.bchemnet.com debian Release.gpg                           
  Get:1 http://www.bchemnet.com debian/extra Translation-en_US
  Hit http://www.bchemnet.com debian Release             
  Get:2 http://www.bchemnet.com debian/extra Packages [13,2kB]
  99% [1 Translation-en_US bzip2 0]

It stays at 99% for hours and never ends ...

Is there something wrong with the repository or with our setup?

Thank you for your help!

[bchemnet]

I don't have any indication of errors with the server.  Perhaps try cleaning out your cache, as described here:
http://www.bchemnet.com/suldr/forum/index.php?topic=67.msg273#msg273

[wolfetti]

I don't have any indication of errors with the server.  Perhaps try cleaning out your cache, as described here:
http://www.bchemnet.com/suldr/forum/index.php?topic=67.msg273#msg273

This guide doesn't work. I've also tryed to download the file from my browser and it seems that the bzip2 translation files are corrupted. No problem with the files in gz format.

[bchemnet]

This guide doesn't work. I've also tryed to download the file from my browser and it seems that the bzip2 translation files are corrupted. No problem with the files in gz format.

What problem are you actually having?  More than 1200 people have successfully downloaded packages in the last 24 hours, so the repository in general is working.

The "translation" files are all empty placeholders, regardless of format.  (They exist simply to avoid generating millions of "file not found" errors each month, which tend to annoy my provider because of the huge log files and the appearance of a denial-of-service attack.)  So it is not possible for one format to be corrupt and another to work, although error messages may indicate otherwise.

[wolfetti]

The problem is the same described in the topic title. Maybe I've forget to say that the problem appears only on fresh installation of the repository, as described here: http://www.bchemnet.com/suldr/ , in the section named "Setting Up the Repository".


ps: sorry for my bad english :)

[bchemnet]

I have no real idea what is causing this.  Perhaps a bad network hop between your computer and the repository?  In the absence of something more specific or additional reports of a problem, I can't see a solution.

[wolfetti]

Maybe you can recreate this files, and solve the problem... However, if you show me a list of test that I can do, I'll be happy to execute them and report the results.

Thank you!

[bchemnet]

There is nothing to recreate - the translation files are zero bytes.  And all files test fine from where I am, and seem to be working for thousands of other people all over the world.  Getting stuck at 99% of a zero byte file doesn't even make sense, which suggests that the real problem is something else.

If you still observe that this only happens with a new setup of the repository and not with an existing older one, perhaps there is some small error in the way the new one was set up?

[wolfetti]

I don't think so, cause I've installed the driver following the same procedure on other machines a few time ago and all the procedure worked. For a detailed explanation, the procedure that I've always followed (in a simple bash script) is this:

#1 - echo "deb http://www.bchemnet.com/suldr/ debian extra" >> /etc/apt/sources.list
#2 - wget -O - http://www.bchemnet.com/suldr/suldr.gpg | apt-key add -
#3 - apt-get update
#4 - apt-get install suld-driver-4.01.17

This workflow fail at point 3 with the never ending translation at 99%.
The only workaround that I've found is to copy the 'www.bchemnet.com_suldr_dists_debian_extra_i18n_*' files from a working machine into /var/lib/apt/lists before run the apt-get update command.

[bchemnet]

Based on what I can find with web searches, the only known solutions to problems of this type are resolved at the user end.  See, for example, http://askubuntu.com/questions/158148/why-does-apt-get-keep-waiting-for-headers-on-oracle-repository (and the links therein), but there are other references if you search for something like "apt get translation 99%".  Most commonly it does require a thorough cleaning of the cache or addressing some sort of network configuration problem with many simultaneous connections.

It could well be the multiple connections issue in your case, because the translation files are static.  Therefore, apt only attempts to download them on a fresh install, and makes many more connections to the server at that time than on subsequent updates.

But copying them from a working computer is equivalent, so you should not encounter this problem again regardless of the actual source.

[mkbrewer]

apt-get update hangs forever on trying to decompress 0 byte files.  This is a known bug in some versions of apt. See: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=595691  My version of apt (0.7.25.3ubuntu9.14) has this bug.  I discovered this when I tried to add a new repository and found that Synaptic couldn't load the package information for it as it hung on trying to download www.bchemnet/suldr/debian/extra Translation-en_US. I also noticed that Update Manager hadn't successfully updated it's indices for 4 months because of this issue.  Unfortunately, I can't upgrade to the latest version of apt as I am locked in to Ubuntu 10.04 LTS until my major software vendor announces support for a later version.

  Bottom line: Unless you can do something about the 0 byte Translation files I must say sayonara to your repository.  It's too bad.  I enjoyed the automatic updates to my printer driver while they lasted.

[bchemnet]

Thanks for sharing this.

I can't eliminate the zero-byte files, for the reasons described by one of the posters to that bug forum.  (I get hits on translation files at an average rate of 2-3 per second and peak rate of 50-100/second - if I let all those be 404 errors, the error log and server load get huge, and my host complains about not being able to detect denial of service attacks through the noise I generate.)  Making them non-zero-byte but still not full translations won't help, and I can't generate translations in over 100 languages.  And generating "fake" translations (e.g., making them all identical English ones) is (a) still much harder on the server and (b) misleading to users who actually aren't using English (a majority).

In my view, this bug should have been accepted and pushed out to 10.04 LTS as a patch to apt.  But since that seems unlikely, the two solutions I see are:

1) Disable translation files (how to do this is also given in the same bug report) (part of the problem here is that around the same time, apt defaulted to always looking for translation files, which creates the noise that leads to the need for zero-byte files.....)

2) Disable this repository.

Given the actual frequency of updates here, option 2 with a periodic (once every 3-6 months or when a problem occurs) re-enable to see if a new driver version is available actually seems perfectly reasonable to me.  It's clear to me from download stats that a sizable fraction of users are only checking for (or applying) updates every few weeks anyway.

[ieri]

The only workaround that I've found is to copy the 'www.bchemnet.com_suldr_dists_debian_extra_i18n_*' files from a working machine into /var/lib/apt/lists before run the apt-get update command.

Thank you wolfetti for the workaround! It works fine!

For those who does not have access to a working machine, I attached an archive with the files to copy in /var/lib/apt/lists.

Once you have the files, you should be able to run "sudo apt-get update" without any timeout.