Outils pour utilisateurs

Outils du site


blog

Draft Python3

List comprehensions and generator expressions

(short form: “listcomps” and “genexps”)

line_list = ['  line 1\n', 'line 2  \n', ' \n', '']
 
# Generator expression -- returns iterator
stripped_iter = (line.strip() for line in line_list)
 
# List comprehension -- returns list
stripped_list = [line.strip() for line in line_list]
 
stripped_list = [line.strip() for line in line_list if line != ""]
Autres
def inc(f, id):
    """ Exemple : inc(lambda x : x+1, 0) """
    try :
        global counter
        counter = f(counter)
    except NameError:
        counter = 0
    return counter

http://mgautier.fr/blog/Astuce/changer-lenvironnement-bash-avec-python.html


List unique remove duplicate

Source : http://stackoverflow.com/questions/89178/in-python-what-is-the-fastest-algorithm-for-removing-duplicates-from-a-list-so

def unique(items):
    found = set([])
    keep = []
    for item in items:
        if item not in found:
            found.add(item)
            keep.append(item)
    return keep

How do you split a list into evenly sized chunks in Python?

Source : http://stackoverflow.com/questions/312443/how-do-you-split-a-list-into-evenly-sized-chunks-in-python

def chunks(l, n):
    n = max(1, n)
    return [l[i:i + n] for i in range(0, len(l), n)]

Determine if variable is defined in Python

http://stackoverflow.com/questions/1592565/determine-if-variable-is-defined-in-python

try:
  thevariable
except NameError:
  print "well, it WASN'T defined after all!"
else:
  print "sure, it was defined."

I think it's better to avoid the situation. It's cleaner and clearer to write:

a = None
if condition:
    a = 42

Getting file size in Python

http://stackoverflow.com/questions/6591931/getting-file-size-in-python

def get_Size(file):
    file.seek(0,2) # move the cursor to the end of the file
    size = file.tell()
    return size

Exemple :

with open('plop.bin','rb') as file:
    size=get_Size(file)
 
with open('plop.bin','rb') as file:
    #size=get_Size(file)
    for i in range(size):
        un, deux = read_hexafile()
        msg.append(un)
        msg.append(deux)

/dev/null

fnull = open(os.devnull, 'w')

Message d'erreur

How to print to stderr in Python?

http://stackoverflow.com/questions/5574702/how-to-print-to-stderr-in-python

def warning(*objs):
    print(*objs, file=sys.stderr)

Conversion hexa

import numpy as np
 
carac=re.sub('^','0x',carac)
carac= int(carac, 16)
print(carac)
file.write(np.byte(carac))
 
 
with open('plop.bin', 'bw') as file:
        file.write(b'\x50\x40\x73')
 
 
hex(11) # '0xb'
 
a = int('0x100', 16)
print(a)   #256
print('%x' % a) #100
 
import binascii
binascii.unhexlify('7061756c')  # b'paul'
 
"{0:8b}".format(int("a",16))    # '    1010'

http://stackoverflow.com/questions/16843108/how-to-read-a-hex-file-into-numpy-array

with open(myfile) as f:
    iv = binascii.unhexlify(f.readline().strip())
    key = binascii.unhexlify(f.readline().strip())
    count = int(f.readline())
    a = np.fromiter((binascii.unhexlify(line.strip()) for line in f), dtype='|S16')
hashlib.sha512('Bonjour'.encode('utf-8')).hexdigest()
hex(int.from_bytes('Bonjour'.encode('utf-8'), 'big'))

Créer un Dictionnaire à partir de deux listes (l'une clef, l'autre valeur)

clef = ['a', 'b', 'c']
valeur = [1, 2, 3]
dictionnaire=dict(zip(clef, valeur))

Test

Voir :

  • nox
  • unittest

Exception

http://stackoverflow.com/questions/16138232/is-it-a-good-practice-to-use-try-except-else-in-python

try:
    s   
except NameError:
    pass
else: # If no exception occured, do :
    s.user.logout()

Temps / time

Voir :

  • timeit
start = time.time()
UnifiedJob.objects.filter(id=1096679).update(status='canceled')
end = time.time()
 
print(end - start)

Strings

Debug

Source : https://www.geekarea.fr/wordpress/?p=763

Level 1

f = open('/tmp/debug','a')
f.write(variable + '\n')
f.close()

Level 2

from pprint import pprint
pprint(variable.__class__.__name__, f)
pprint(dir(variable), f)
pprint(vars(variable), f)

Level 3 (sur une exception)

import traceback
f.write(str(traceback.format_exc()))

map reduce filter

Aures

A noter que sur RedHat 8 le chemin vers python est /usr/libexec/platform-python

2025/03/24 15:06

Python virtualenv pyvenv pyenv pipenv poetry

pyvenv

Remplacé par python3 -m venv

pyvenv plop
cd plop
source bin/activate
pip install --upgrade pip
#deactivate
pyvenv
cd env/
cd ..
source env/bin/activate
pip install -r requirements.txt 
python3 -m venv plop
source plop/bin/activate

pipenv

Source : http://sametmax.com/pipenv-solution-moderne-pour-remplacer-pip-et-virtualenv/

Préciser la version de Python à utiliser

python2 --two
python3 --three

Install via PIP

python3 -m pip install pip --upgrade --user
python3 -m pip install pipenv --user

Install via apt

apt-get install pipenv

Exemple

pipenv run python
pipenv shell
pipenv install --dev
# Créer le fichier des dépendances
pipenv lock
 
# Installer toutes les dépendances
pipenv install

Afficher les dépendances

pipenv graph

Chemin du dossier du virtualenv

pipenv --venv
Pb
AttributeError: module 'pip' has no attribute 'get_installed_distributions'
$ pipenv check                                                                                                            
Checking PEP 508 requirements…
Passed!
Checking installed package safety…
An error occurred:
Traceback (most recent call last):
  File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/usr/lib/python3/dist-packages/pipenv/patched/safety.zip/__main__.py", line 8, in <module>                                                                     
  File "/usr/lib/python3/dist-packages/pipenv/patched/safety.zip/click/core.py", line 722, in __call__                                                                 
  File "/usr/lib/python3/dist-packages/pipenv/patched/safety.zip/click/core.py", line 697, in main                                                                     
  File "/usr/lib/python3/dist-packages/pipenv/patched/safety.zip/click/core.py", line 1066, in invoke                                                                  
  File "/usr/lib/python3/dist-packages/pipenv/patched/safety.zip/click/core.py", line 895, in invoke                                                                   
  File "/usr/lib/python3/dist-packages/pipenv/patched/safety.zip/click/core.py", line 535, in invoke                                                                   
  File "/usr/lib/python3/dist-packages/pipenv/patched/safety.zip/safety/cli.py", line 52, in check                                                                     
AttributeError: module 'pip' has no attribute 'get_installed_distributions'
Solution
sudo apt-get -y autoremove --purge pipenv python3-virtualenv-clone
$ python3 -m pip install pip --upgrade --user
$ python3 -m pip install pipenv --user
2025/03/24 15:06

Python tests unitaires

Voir aussi:

Source : http://sametmax.com/un-gros-guide-bien-gras-sur-les-tests-unitaires-en-python-partie-3/

Voir aussi les différents types de test :

  • Tests unitaires (au format unittest ou pytest) : units
  • Tests d’intégration : integration
  • Tests syntaxique : sanity
#pip3 install pytest
apt-get install python3-pytest

Voir aussi : Testinfra

test_plop.py

#! /usr/bin/python3
# coding: utf-8
 
#import pytest
 
from func import plop
 
def test_plop():
    assert foo == bar
pytest-3

Autre

~/swift/.unittests

#!/bin/bash
 
TOP_DIR=$(dirname $(realpath "$0"))
 
cd $TOP_DIR/test/unit
nosetests --exe --with-coverage --cover-package swift --cover-erase --cover-branches --cover-html --cover-html-dir="$TOP_DIR"/cover $@
rvalue=$?
rm -f .coverage
cd -
exit $rvalue
$ dpkg -S /usr/bin/nosetests
python-nose: /usr/bin/nosetests
2025/03/24 15:06

Note Python pip

Voir aussi :

Do not run pip install as root (or with sudo)

Installation dans ~/.local/bin/

easy_install --user pip

Liste des “packages” installés dans la sessions de l'utilisateur :

pip freeze --local

Ou

python -m ensurepip

Voir aussi pipreqs

Installer un package depuis un dépôt git

pip install --user git+https://github.com/benoit-intrw/livestreamer

Mise à jour des “packages” locaux

pip install --local -U livestreamer

Installation d'un “package” dans la session de l'utilisateur (pas besoin de droit root) (pas besoin de prendre le risque d’abîmer sa distro)

pip install --user livestreamer

Tous mettre à jour localement

pip freeze --local | grep -v '^\-e' | cut -d = -f 1  | xargs -n1 pip install -U

Source http://stackoverflow.com/questions/2720014/upgrading-all-packages-with-pip

Fichier requirements.txt

env1/bin/pip freeze > requirements.txt
env2/bin/pip install -r requirements.txt

Voir aussi pipreqs

Exemple de fichier requirements.txt

requirements.txt

numpy>=1.15.0
opencv-python>=3.4.2.17
pandas>=0.23.3 --no-binary pandas
Pillow>=5.2.0
tensorflow==1.8.0
tensorflow-gpu==1.8.0

Variable Python - utiliser les dist-packages depuis un environnement virtuel

~/.bashrc

export PYTHONPATH=$PYTHONPATH:/usr/lib/python3/dist-packages/:$HOME/test/lib/python3.5/site-packages/

Utiliser /usr/local

~/.config/pip/pip.conf

[global]
target = /usr/local/lib/python2.7/site-packages

Ou

PIP_TARGET=/usr/local/lib/python2.7/site-packages

~/.bashrc

export PYTHONPATH=$PYTHONPATH:/usr/local/lib/python2.7/site-packages

Exemple de PIP en interne

pip.conf

[global]
timeout = 300
index-url = https://pypi.acme.fr/pypi/+simple/
cert = /etc/ssl/certs/ca-certificates.crt

pip.conf

mkdir -p ~/.pip/ && \
echo "[global]" >> ~/.pip/pip.conf && \
URLPIP="https://artifactory.packages.install-os.acme.local/api/pypi/ext_pypi/simple" && \
echo "index = $URLPIP " >> ~/.pip/pip.conf && \
echo "index-url = $URLPIP " >> ~/.pip/pip.conf && \
echo "cert = /etc/ssl/certs/ca-certificates.crt " >> ~/.pip/pip.conf 

Autres

ssh -R3128:192.168.56.1:3128 user@192.168.205.11
 
export http_proxy=http://127.0.0.1:3128
export https_proxy=http://127.0.0.1:3128
 
python3 -m venv test
cd test
source bin/activate
pip install -U pip
# pip install --use-feature=2020-resolver plop
pip install --no-binary pandas pandas
#deactivate
export PIP_BREAK_SYSTEM_PACKAGES=1

Voir mise_a_jour_repos_git

constraints

You can achieve this with a constraints file. Just put all your constraints into that file:

constraints.txt

google-api-core==1.16.0

Then you can install via:

python -m pip install -c constraints.txt google-cloud-secret-manager

This will try every version of google-cloud-secret-manager, starting from the most recent version, until it finds a version that is compatible with the given constraints.

pipx

python3 -m pip install --user pipx
python3 -m pipx ensurepath
2025/03/24 15:06

Cluster Pacemaker et Corosync partition GFS2

Brouillon

Voir :

Voir aussi :

/sbin/lvmconf --enable-cluster

ou

/etc/lvm/lvm.conf

locking_type = 3
sed -i -e "s/use_lvmetad \= 1/use_lvmetad \= 0/g" /etc/lvm/lvm.conf
systemctl stop lvm2-lvmetad.service
dracut -H -f /boot/initramfs-$(uname -r).img  $(uname -r)
yum install rgmanager lvm2-cluster gfs2-utils
 
mkfs.gfs2 -p lock_dlm -t my_cluster:FSName -j 2 /dev/sdb

-j 2
2 journaux car deux nœuds

Teste

mount -t gfs2 -o locktable="my_cluster:FSName" /dev/sdb /mnt/gfs/
pcs resource create dlm ocf:pacemaker:controld op monitor interval=30s on-fail=fence clone interleave=true ordered=true
 
pcs resource create clvmd ocf:heartbeat:clvm op monitor interval=30s on-fail=fence clone interleave=true ordered=true
 
pcs constraint order start dlm-clone then clvmd-clone
pcs constraint colocation add clvmd-clone with dlm-clone
pcs resource create clusterfs Filesystem device="/dev/vg_data/lv_test1" directory="/mnt/gfs" fstype="gfs2" "options=noatime" \
op monitor interval=10s on-fail=fence clone interleave=true

Pb

Transport endpoint is not connected
mount: mount /dev/sdb on /mnt/gfs failed: Transport endpoint is not connected

Solution

systemctl start dlm
systemctl enable dlm
2025/03/24 15:06
blog.txt · Dernière modification : de 127.0.0.1

Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki