Installation issues packages hash sum mismatch

Hi,
Starting today we get errors when trying to install packages from couchbase repository.
we get error when running apt-get update:

W: Failed to fetch http://packages.couchbase.com/ubuntu/dists/trusty/trusty/main/binary-amd64/Packages  Hash Sum 
mismatch

I found earlier threads that report the same issue after version upload from your side and i can see that yesterday libcouchbase package updated.
(Installation issues (packages) - Hash Sum mismatch)

I tried to clean /var/lib/apt/lists/, run apt-get clean and update without success

can you please check ?

We put our releases to s3, and have cloudfront configured. Most likely it hasn’t been invalidated yet. Could you post output of the following command?

$ curl -I http://packages.couchbase.com/ubuntu/dists/trusty/trusty/main/binary-amd64/Packages
HTTP/1.1 200 OK
Content-Type: binary/octet-stream
Content-Length: 5033
Connection: keep-alive
Date: Wed, 12 Jul 2017 18:09:28 GMT
Last-Modified: Wed, 12 Jul 2017 09:29:28 GMT
ETag: "217ce5fd420630a7526e0a95839afee2"
x-amz-version-id: 2ITjvXvgSikfrC_j87VYfqW9tjYhJ_qG
Accept-Ranges: bytes
Server: AmazonS3
X-Cache: Miss from cloudfront
Via: 1.1 2d1245882c0beb52d08e23e1fa6ab592.cloudfront.net (CloudFront)
X-Amz-Cf-Id: nPTBcgVpXuKVs1tJxYE5qvQPEplu1hXOck_fAuHfI_dins4Sz3Pj-A==

thanks for the quick reply

HTTP/1.1 200 OK
Content-Type: binary/octet-stream
Content-Length: 5033
Connection: keep-alive
Date: Wed, 12 Jul 2017 09:45:20 GMT
Last-Modified: Wed, 12 Jul 2017 09:29:28 GMT
ETag: "217ce5fd420630a7526e0a95839afee2"
x-amz-version-id: 2ITjvXvgSikfrC_j87VYfqW9tjYhJ_qG
Accept-Ranges: bytes
Server: AmazonS3
Age: 30317
X-Cache: Hit from cloudfront
Via: 1.1 85359b00dd31672f055ad3cc68c51eff.cloudfront.net (CloudFront)
X-Amz-Cf-Id: parGh4WD6i62NBp_vp95YVgN22N0UDFAc9TG6J2BiIGYu4C62syOZw==

and this one?

$ curl -I http://packages.couchbase.com/ubuntu/dists/trusty/trusty/main/binary-amd64/Packages.gz
HTTP/1.1 200 OK
Content-Type: binary/octet-stream
Content-Length: 1454
Connection: keep-alive
Date: Thu, 06 Jul 2017 13:57:16 GMT
Last-Modified: Wed, 17 May 2017 19:33:00 GMT
ETag: "8c995f205c859292158464e1b3e243bc"
x-amz-meta-s3cmd-attrs: uid:501/gname:staff/uname:mnunberg/gid:20/mode:33188/mtime:1495047245/atime:1495047245/md5:8c995f205c859292158464e1b3e243bc/ctime:1495049513
x-amz-version-id: Kd.4R9hl.wi4csC.cLEuX0KPsRhvq2hr
Accept-Ranges: bytes
Server: AmazonS3
Age: 71768
X-Cache: Hit from cloudfront
Via: 1.1 0e6067b46ed4b3e688f898d03e5c1c67.cloudfront.net (CloudFront)
X-Amz-Cf-Id: wPH-63Ey5UhnelPYVHg-Fao_OSKVh-g3hXjROy0UdPd3kZGVW4AuyA==

From my place it seems like stale

both hit cloudfront

HTTP/1.1 200 OK
Content-Type: binary/octet-stream
Content-Length: 1454
Connection: keep-alive
Date: Thu, 18 May 2017 01:16:34 GMT
Last-Modified: Wed, 17 May 2017 19:33:00 GMT
ETag: "8c995f205c859292158464e1b3e243bc"
x-amz-meta-s3cmd-attrs: uid:501/gname:staff/uname:mnunberg/gid:20/mode:33188/mtime:1495047245/atime:1495047245/md5:8c995f205c859292158464e1b3e243bc/ctime:1495049513
x-amz-version-id: Kd.4R9hl.wi4csC.cLEuX0KPsRhvq2hr
Accept-Ranges: bytes
Server: AmazonS3
Age: 60213
X-Cache: Hit from cloudfront
Via: 1.1 72f0ac9702110cafbb646d71a297e2c7.cloudfront.net (CloudFront)
X-Amz-Cf-Id: ouzNl38UFgnwhI_0ikJbIRTERZK5ep5UoWNUWyYp3WEj5Lf9xPVjfQ==

yes, but notice Last-Modified tag of the second. I will try to contact our support team about this question

Hi there,

I have the same issue… again! :wink: I saw you linked my last topic - that’s good because it should be noticed by your support team @avsej. And really, this is a few hours of downtime when I cannot rebuild by environments. It would be good to receive any notice or message few days before this kind of release at least. Or just do something to avoid situations like this. Thanks a lot!

Maciej

Additionally we publish tarballs where all DEBs and RPMs bundled:

http://packages.couchbase.com/clients/c/libcouchbase-2.7.6_centos62_i686.tar
http://packages.couchbase.com/clients/c/libcouchbase-2.7.6_centos62_x86_64.tar
http://packages.couchbase.com/clients/c/libcouchbase-2.7.6_centos7_x86_64.tar
http://packages.couchbase.com/clients/c/libcouchbase-2.7.6_jessie_amd64.tar
http://packages.couchbase.com/clients/c/libcouchbase-2.7.6_jessie_i386.tar
http://packages.couchbase.com/clients/c/libcouchbase-2.7.6_ubuntu1204_amd64.tar
http://packages.couchbase.com/clients/c/libcouchbase-2.7.6_ubuntu1204_i386.tar
http://packages.couchbase.com/clients/c/libcouchbase-2.7.6_ubuntu1404_amd64.tar
http://packages.couchbase.com/clients/c/libcouchbase-2.7.6_ubuntu1404_i386.tar
http://packages.couchbase.com/clients/c/libcouchbase-2.7.6_ubuntu1604_amd64.tar
http://packages.couchbase.com/clients/c/libcouchbase-2.7.6_ubuntu1604_i386.tar
http://packages.couchbase.com/clients/c/libcouchbase-2.7.6_wheezy_amd64.tar
http://packages.couchbase.com/clients/c/libcouchbase-2.7.6_wheezy_i386.tar

@lior.b, @maciej.tomaszewski, everything should be good now. And next time I will invalidate CloudFront during upload. Sorry for inconvenience.

@avesj Started seeing this again today. Is this the same issue?

Failed to fetch http://packages.couchbase.com/ubuntu/dists/xenial/xenial/main/binary-amd64/Packages.gz Hash Sum mismatch

Don’t worry, it will disappear soon. This is because of bad design of APT repositories and the way CloudFront caches data. It will be fixed once CloudFront will receive updated versions of the index files.

@avsej appears we’re seeing this issue today. Was curious on how long the cache typically stays up.

Failed to fetch http://packages.couchbase.com/releases/couchbase-server/community/deb/dists/xenial/xenial/main/binary-amd64/Packages.gz Hash Sum mismatch