curl –connect-timeout and –max-time does not work as expected
Problem Description:
The following command will wait > 60 seconds on Ubuntu 22.04:
curl --verbose --retry-max-time 0 --retry 0 --connect-timeout "30" --max-time "60" "https://www.google.com/"
Here is the testing result:
[email protected]:~# echo $(date)
Tue Dec 6 07:26:04 PM CST 2022
[email protected]:~# curl --verbose --retry-max-time 0 --retry 0 --connect-timeout "30" --max-time "60" "https://www.google.com/"
* Resolving timed out after 30000 milliseconds
* Closing connection 0
curl: (28) Resolving timed out after 30000 milliseconds
[email protected]:~# echo $(date)
Tue Dec 6 07:28:26 PM CST 2022
Here is the version:
[email protected]:~# curl --version
curl 7.81.0 (x86_64-pc-linux-gnu) libcurl/7.81.0 OpenSSL/3.0.2 zlib/1.2.11 brotli/1.0.9 zstd/1.4.8 libidn2/2.3.2 libpsl/0.21.0 (+libidn2/2.3.2) libssh/0.9.6/openssl/zlib nghttp2/1.43.0 librtmp/2.3 OpenLDAP/2.5.12
Release-Date: 2022-01-05
Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS brotli GSS-API HSTS HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz NTLM NTLM_WB PSL SPNEGO SSL TLS-SRP UnixSockets zstd
The expected behavior:
The whole timeout should not exceed the 60 seconds or 30 seconds as it
already timeout in Resolving DNS.
What’s the problem and how to fix the timeout with curl?
Solution – 1
It is possible that DNS is not resolving properly and causing cURL to take longer. The issue you describe, looking at your cURL version seems to be similar to what is below:
https://unix.stackexchange.com/questions/571246/curl-max-time-and-connect-timeout-not-working-at-all
Instead of cracking your head with rebuilding cURL and configuring it with c-ares, it might be worth a try to test whether a server is reachable first and then initiate cURL:
ping -c 1 google.com &>/dev/null && cURL ...whatever
By default ping will extend timeout by max 4 seconds, which you can finetune using -W option