Voir :
URI (Uniform Resource Identifier) :
URL, URI and URN
En anglais, on la décrit ainsi :
scheme : // authority / path ? query # hash
Dans sa traduction française :
plan : // autorité / chemin ? requête # fragment
Enfin , comme on l'explique à notre manière :
comment : // chez_qui / où ? question # extrait
Source : https://cpu.dascritch.net/post/2023/10/12/Standard-%3A-%28Rappel%29-les-segments-d-une-URL
Syntax of URL
scheme: subdomain/domain-name.Top-level-domain/sub-folder
curl as wget
curl -O -L http://acme.org/download/plop.tar.gz
Point wget to TOR/Privoxy:
#export http_proxy=http://127.0.0.1:8118/ #export https_proxy=http://127.0.0.1:8118/ export ALL_PROXY=http://127.0.0.1:8118/ # Ou curl -x "http://127.0.0.1:8118" "http://gnu.org"
Voir aussi HTTPie
Voir Utiliser les REST API JSON avec cURL
JSON
curl -k https://aap-controller.local/api/v2/users/27/roles/ -X POST -u user:password --data-raw '{"id":534}'
Préciser le host sans modifier les /etc/hosts
curl -H "Host:www.acme.fr" 192.168.2.14:80
Meteo (Basé sur Wego)
curl -4 wttr.in curl -H "Accept-Language: it" wttr.in/Rome
/etc/hosts
curl --resolve www.acme.fr:80:192.168.2.10 \ --resolve www.acme.fr:443:192.168.2.10 \ http://www.acme.fr
Forcer SSLv3 Pour vérifier que ça ne marche pas, car ca chiffrement est obsolète -k Pour les certificat auto-signé
Voir CURLOPT_SSLVERSION
curl -3 -k --show-error --verbose https://acme.fr
Erreur status code
curl --show-error --verbose -I http://www.acme.fr curl --write-out "\nhttp_code=%{http_code}\ total_time=%{time_total}\n"
Add GPG key
curl -fsSL https://download.docker.com/linux/debian/gpg | sudo apt-key add -
-f : Fail silently on server errors
-s : Silent or quiet mode. Don't show progress meter or error messages
-S : When used with -s, --silent, it makes curl show an error message if it fails
-L : If the server reports that the requested page has moved to a different location, this option will make curl redo the request on the new place.
.curlrc
# Source : https://gist.github.com/v1m/f1d4751883f19c916515 # store the trace in curl_trace.txt file. beware that multiple executions of the curl command will overwrite this file --trace curl_trace.txt # store the header info in curl_headers.txt file. beware that multiple executions of the curl command will overwrite this file --dump-header curl_headers.txt #change the below referrer URL or comment it out entirely -e "https://www.google.com" #change the below useragent string. get your/other UA strings from http://www.useragentstring.com/ -A "Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/525.13 (KHTML, like Gecko) Chrome/0.2.149.27 Safari/525.13" #some headers -H "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8" -H "Upgrade-Insecure-Requests: 1" -H "Accept-Encoding: gzip, deflate, sdch" -H "Accept-Language: en-US,en;q=0.8"
.curlrc
Source https://github.com/necolas/dotfiles/blob/0d402c916f6e1118b20bd4befff863c1e4a2ee2d/shell/curlrc # Limit the time (in seconds) the connection to the server is allowed to take connect-timeout = 30 # Limit the time (in seconds) the whole operation is allowed to take # (prevents cURL from hanging due to slow networks or links going down) max-time = 300 # When following a redirect, automatically set the previous URL as referer referer = ";auto" # cURL default configuration file # http://curl.haxx.se/docs/manpage.html # Show error messages show-error # Show progress using a progress meter progress-bar # Send a fake UA string for the HTTP servers that sniff it user-agent = "Mozilla/5.0 Gecko"
Upload
HTTP POST files here:
curl --proxy 127.0.0.1:4444 -F'file=@yourfile.png' http://0xff.i2p
Or with status bar:
curl --proxy 127.0.0.1:4444 -T yourfile.png http://0xff.i2p | tee
Upload command output:
command | curl -T - --proxy 127.0.0.1:4444 http://0xff.i2p
Upload command output with stderr:
command 2>&1 | curl -T - --proxy 127.0.0.1:4444 http://0xff.i2p
Or you can shorten URLs:
curl --proxy 127.0.0.1:4444 -F'shorten=http://example.com/some/long/url' http://0xff.i2p
Autres
wget --no-cache --no-cookie
wget https://getsession.org/linux # Saving to: 'linux' wget --content-disposition https://getsession.org/linux # Saving to: 'session-desktop-linux-x86_64-1.8.6.AppImage' wget -O plop https://getsession.org/linux # Saving to: ‘plop'
Fix curl client certificate error: curl: (58) unable to set private key file http://honglus.blogspot.fr/2012/03/fix-curl-client-certificate-error-curl.html
15 Practical Linux cURL Command Examples (cURL Download Examples) http://www.thegeekstuff.com/2012/04/curl-examples/
Download the webpage with wget. The wget options (-E, -k, etc.) are CASE SENSITIVE! The maximum depth to which the retrieval may descend is specified with the ‘-l’ option:
wget -E -k -p -nd -r -l 1 -np -P <local-folder-name> http://site-to-be-mirrored.org
wget : Ignore robots restrictions
wget -e robots=off --wait 1 http://your.site.here
Dossier répertoire directory
wget http://www.claudiokuenzler.com/nagios-plugins/check_equallogic.sh -P /usr/lib/nagios/plugins/
A partir d'un fichier d'URL
wget --no-proxy -i providers.txt -P /usr/share/terraform/providers/registry.terraform.io/hashicorp/vsphere
wget
wget --max-redirect=4 --output-document=siteexport.zip --post-data="id=dokuwiki&depth=0&depthType=0.0&do=siteexport&ens=dokuwiki&template=dokuwiki" https://acme.com/doku.php?id=doku.php --http-user=USER --http-passwd=PASSWD
wget -E -r http://wiki.foo.tld/doku.php?id=start -l 0 --header="X-DokuWiki-Do: export_xhtml"
curl
curl -L --max-redirs 46 -o siteexport.zip -d "id=dokuwiki&depth=0&depthType=1.0&do=siteexport&ens=dokuwiki&exportbody=1&renderer=&template=dokuwiki" https://acme.com/doku.php?id=doku.php --anyauth --user USER:PASSWD
Pour ignoré le certificat
curl -k https://www.acme.fr wget --no-check-certificate https://www.acme.fr
Pour enregistrer le certificat et vérifier
openssl s_client -showcerts -connect www.acme.fr:443 -servername www.acme.fr </dev/null > cacert.pem # ou curl -k -w %{certs} https://www.acme.fr > cacert.pem
Tester
curl --cacert cacert.pem https://www.acme.fr
ou avec wget
wget --ca-certificate=cert.pem https://www.acme.fr
Voir Installation de la CA sous GNU/Linux
Enlever la CA
rm /usr/local/share/ca-certificates/acme.fr.crt update-ca-certificates --fresh
TODO : equivalement pour Redhat avec update-ca-trust
.curlrc
cacert=/home/jean/cert.pem
Ou, à éviter mais possible .curlrc
insecure
https sous java Voir la paquet ca-certificates-java
openssl s_client -showcerts -connect 192.168.4.13:5000 </dev/null > cert.crt # test curl --show-error --verbose --cacert cert.crt https://172.18.51.220:5000 cp cert.crt /usr/local/share/ca-certificates/acme.fr.crt update-ca-certificates /var/lib/dpkg/info/ca-certificates-java.postinst configure systemctl restart jenkins
import requests # No warn for bad https 'verify=False' requests.packages.urllib3.disable_warnings() # HTTP GET all users ids r = requests.get('https://aap.admin.acme.local/api/v2/users/', auth=(aap_user, aap_pass), verify=False)
curl --verbose --show-error --tlsv1.2 --cert /etc/httpd/ssl/acme.fr.crt --key /etc/httpd/ssl/acme.fr.key https://acme.net/plop wget --debug --verbose --no-check-certificate --secure-protocol=TLSv1_2 --certificate=/etc/httpd/ssl/acme.fr.crt --private-key=/etc/httpd/ssl/acme.fr.key https://acme.net/plop (echo -ne "GET /plop HTTP/1.1\r\nHost: acme.net\r\n\r\n" ; cat ) |openssl s_client -connect acme.net:443 -servername acme.net -cert /etc/httpd/ssl/acme.fr.crt -key /etc/httpd/ssl/acme.fr.key
curl --cacert keys/ca.crt --key keys/client1.key --cert keys/client1.crt https://pkiweb.lan wget --ca-certificate=keys/ca.crt --private-key=keys/client1.key --certificate=keys/client1.crt https://pkiweb.lan
# Your file will be named myfile.txt curl -F file=@myfile.txt https://up.depado.eu/ # Your file will be named myamazingfile! curl -F name="myamazingfile!" -F file=@myfile.txt https://up.depado.eu/ # Reads data from stdin cat myfile.txt | curl -F name="stdin" -F file=@_ https://up.depado.eu/
Pipe bash
https://www.idontplaydarts.com/2016/04/detecting-curl-pipe-bash-server-side/
curl -sSk http://pluver.xqi.cc/setup.bash curl -sSk http://pluver.xqi.cc/setup.bash |bash