Difference between revisions of "Python"

From Got Opinion Wiki
Jump to navigation Jump to search
m (added BeeWare)
 
(21 intermediate revisions by the same user not shown)
Line 10: Line 10:


[[My Anaconda notes]]
[[My Anaconda notes]]
=== Python virtual environments ===
'''Use this'''
[https://docs.python.org/3/library/venv.html Python 3 docs]
'''Don't use the following'''
[https://python-guide-cn.readthedocs.io/en/latest/dev/virtualenvs.html Python venv tutorial]
Setup Python virtual environment on Windows 10 using cmder in directory pydub
<pre>C:\Users\anon\Documents\python-venv         
λ python -m venv pydub                       
                                             
C:\Users\anon\Documents\python-venv         
λ ls -l                                     
total 0                                     
drwxr-xr-x 1 anon 197121 0 Nov 26 15:37 pydub/</pre>
Activate virtual environment. If you encounter an error about policy not allowing script, run <code>Set-ExecutionPolicy Unrestricted -Scope Process</code> then run activate script again.
<pre>C:\Users\anon\Documents\python-venv
λ .\pydub\Scripts\activate
C:\Users\anon\Documents\python-venv
(pydub) λ</pre>
Deactivate virtual environment
<pre>C:\Users\anon\Documents\python-venv
(pydub) λ deactivate
C:\Users\anon\Documents\python-venv
λ</pre>
To delete virtual environment just delete the directory used in <code>python -m venv</code> command (pydub in my example)


== Python resources ==
== Python resources ==


=== Online ===
=== Online ===
[https://www.delftstack.com/howto/python/ Collection of Python HOWTOs]


[https://pymotw.com/3/index.html Python 3 Module of the Week] by [http://doughellmann.com/ Doug Hellmann] (PyMOTW-3)
[https://pymotw.com/3/index.html Python 3 Module of the Week] by [http://doughellmann.com/ Doug Hellmann] (PyMOTW-3)
Line 22: Line 61:


[https://christopherdavis.me/blog/threading-basics.html Threading basics]
[https://christopherdavis.me/blog/threading-basics.html Threading basics]
[https://realpython.com/python-async-features/ Python Async Features]
[https://www.freecodecamp.org/news/learn-python-by-building-5-games/ Learn Python by building 5 games]
[http://python-notes.curiousefficiency.org/en/latest/index.html Nick Coghlan’s Python Notes]
[http://docs.python-guide.org/en/latest/ Python guide]
Open book [http://openbookproject.net/thinkcs/Think Like a Computer Scientist]
Open book [http://www.openbookproject.net/pybiblio/ Python Bibliotheca]
[https://realpython.com/courses/arduino-python/ Arduino With Python: How to Get Started] '''(video)'''


=== Books ===
=== Books ===
Line 28: Line 81:


[http://gnosis.cx/publish/programming/regular_expressions.html Text Processing in Python]
[http://gnosis.cx/publish/programming/regular_expressions.html Text Processing in Python]
=== Courses ===
[https://www.freecodecamp.org/news/learn-data-analysis-with-python-course/ Learn Data Analysis with Python – A Free 4-Hour Course]


== Python Frameworks ==
== Python Frameworks ==
Line 58: Line 115:


[http://matplotlib.org/ matplotlib] is a python 2D plotting library which produces publication quality figures in a variety of hardcopy formats and interactive environments across platforms. matplotlib can be used in python scripts, the python and ipython shell, web application servers, and six graphical user interface toolkits.
[http://matplotlib.org/ matplotlib] is a python 2D plotting library which produces publication quality figures in a variety of hardcopy formats and interactive environments across platforms. matplotlib can be used in python scripts, the python and ipython shell, web application servers, and six graphical user interface toolkits.
=== My pathlib notes ===
[https://medium.com/@ageitgey/python-3-quick-tip-the-easy-way-to-deal-with-file-paths-on-windows-mac-and-linux-11a072b58d5f Easy way to deal with Windows file paths]
=== My Python audio notes ===
[https://www.tutorialspoint.com/read-and-write-wav-files-using-python-wave Read and write wav files using Python wave module]
==== pydub notes ====
Install pydub
<pre>C:\Users\anon\Documents\python-venv
(pydub) λ pip install pydub</pre>
Install simpleaudio
<pre>C:\Users\anon\PycharmProjects\pydub
(pydub) λ pip install --upgrade pip setuptools
C:\Users\anon\PycharmProjects\pydub
(pydub) λ pip install simpleaudio</pre>
Sample Python script for splitting wav files on silence
<pre>import os
import argparse
import re
from pydub import AudioSegment
from pydub.silence import split_on_silence
parser = argparse.ArgumentParser(description='Split .wav files at one second or more of silence. A user specified '
                                            'directory will be searched and files with .wav extension will be '
                                            'processed. Split .wav files are stored in "chunks" directory relative to '
                                            'and with similar name as reference .wav file. Split files have "_NN" '
                                            'inserted into file name before .wav extension where NN is 00 to 99 '
                                            'incremented from 00. Supports up to 100 chunks per .wav file by only '
                                            'processing the first 100 chunks.')
parser.add_argument('dir', help='the path to directory where .wav files are located')
args = parser.parse_args()
folder_name = args.dir
regex_pattern_wav_file_ext = re.compile(r"""\.wav$""")  # Regex that can identify files ending with .wav.
chunk_path = r'{}\{}'.format(folder_name, 'chunks')
if not os.path.exists(chunk_path) and not os.path.isdir(chunk_path):
    os.mkdir(chunk_path)
for filename in os.listdir(folder_name):
    if regex_pattern_wav_file_ext.search(r'{}\{}'.format(folder_name, filename)):
        wav_file = AudioSegment.from_wav(r'{}\{}'.format(folder_name, filename))
        chunks = split_on_silence(wav_file, min_silence_len=1000, silence_thresh=-60, keep_silence=500)
        if chunks:
            for i, chunk in enumerate(chunks):
                if i > 99:
                    break
                chunk.export(r"{}\chunks\{}_{:02d}.wav".format(folder_name, filename[:-4], i), format="wav")
print('Script completed.')
</pre>


== Data generation ==
== Data generation ==

Latest revision as of 08:31, 4 September 2024

My Python Notes

Python programming notes

Python programming

My notes using Python libraries

My Python Network Programming Notes

My Anaconda notes

Python virtual environments

Use this

Python 3 docs

Don't use the following

Python venv tutorial

Setup Python virtual environment on Windows 10 using cmder in directory pydub

C:\Users\anon\Documents\python-venv           
λ python -m venv pydub                        
                                              
C:\Users\anon\Documents\python-venv           
λ ls -l                                       
total 0                                       
drwxr-xr-x 1 anon 197121 0 Nov 26 15:37 pydub/

Activate virtual environment. If you encounter an error about policy not allowing script, run Set-ExecutionPolicy Unrestricted -Scope Process then run activate script again.

C:\Users\anon\Documents\python-venv
λ .\pydub\Scripts\activate

C:\Users\anon\Documents\python-venv
(pydub) λ

Deactivate virtual environment

C:\Users\anon\Documents\python-venv
(pydub) λ deactivate
C:\Users\anon\Documents\python-venv
λ

To delete virtual environment just delete the directory used in python -m venv command (pydub in my example)

Python resources

Online

Collection of Python HOWTOs

Python 3 Module of the Week by Doug Hellmann (PyMOTW-3)

PyMOTW-3 Concurrency with Processes, Threads, and Coroutines

Using % and .format() for great good!

Threading basics

Python Async Features

Learn Python by building 5 games

Nick Coghlan’s Python Notes

Python guide

Open book Like a Computer Scientist

Open book Python Bibliotheca

Arduino With Python: How to Get Started (video)

Books

Dive Into Python

Text Processing in Python

Courses

Learn Data Analysis with Python – A Free 4-Hour Course

Python Frameworks

My Django notes

Django is a high-level Python Web framework that encourages rapid development and clean, pragmatic design.

Set up Django in virtual environment on Dreamhost

Django book

BeeWare Write once. Deploy everywhere. Write your apps in Python and release them on iOS, Android, Windows, MacOS, Linux, Web, and tvOS using rich, native user interfaces. Multiple apps, one codebase, with a fully native user experience on every platform.

Libraries

NumPy is the fundamental package for scientific computing with Python. It contains among other things:

  • a powerful N-dimensional array object
  • sophisticated (broadcasting) functions
  • tools for integrating C/C++ and Fortran code
  • useful linear algebra, Fourier transform, and random number capabilities

Besides its obvious scientific uses, NumPy can also be used as an efficient multi-dimensional container of generic data. Arbitrary data-types can be defined. This allows NumPy to seamlessly and speedily integrate with a wide variety of databases.

Twisted is an event-driven networking engine written in Python and licensed under the open source ​MIT license. Twisted runs on Python 2 and an ever growing subset also works with Python 3.

Data Visualization Libraries

Bokeh is a Python interactive visualization library that targets modern web browsers for presentation. Its goal is to provide elegant, concise construction of novel graphics in the style of D3.js, but also deliver this capability with high-performance interactivity over very large or streaming datasets. Bokeh can help anyone who would like to quickly and easily create interactive plots, dashboards, and data applications.

matplotlib is a python 2D plotting library which produces publication quality figures in a variety of hardcopy formats and interactive environments across platforms. matplotlib can be used in python scripts, the python and ipython shell, web application servers, and six graphical user interface toolkits.

My pathlib notes

Easy way to deal with Windows file paths

My Python audio notes

Read and write wav files using Python wave module

pydub notes

Install pydub

C:\Users\anon\Documents\python-venv
(pydub) λ pip install pydub

Install simpleaudio

C:\Users\anon\PycharmProjects\pydub
(pydub) λ pip install --upgrade pip setuptools

C:\Users\anon\PycharmProjects\pydub
(pydub) λ pip install simpleaudio

Sample Python script for splitting wav files on silence

import os
import argparse
import re
from pydub import AudioSegment
from pydub.silence import split_on_silence

parser = argparse.ArgumentParser(description='Split .wav files at one second or more of silence. A user specified '
                                             'directory will be searched and files with .wav extension will be '
                                             'processed. Split .wav files are stored in "chunks" directory relative to '
                                             'and with similar name as reference .wav file. Split files have "_NN" '
                                             'inserted into file name before .wav extension where NN is 00 to 99 '
                                             'incremented from 00. Supports up to 100 chunks per .wav file by only '
                                             'processing the first 100 chunks.')
parser.add_argument('dir', help='the path to directory where .wav files are located')
args = parser.parse_args()
folder_name = args.dir

regex_pattern_wav_file_ext = re.compile(r"""\.wav$""")  # Regex that can identify files ending with .wav.

chunk_path = r'{}\{}'.format(folder_name, 'chunks')
if not os.path.exists(chunk_path) and not os.path.isdir(chunk_path):
    os.mkdir(chunk_path)

for filename in os.listdir(folder_name):
    if regex_pattern_wav_file_ext.search(r'{}\{}'.format(folder_name, filename)):
        wav_file = AudioSegment.from_wav(r'{}\{}'.format(folder_name, filename))
        chunks = split_on_silence(wav_file, min_silence_len=1000, silence_thresh=-60, keep_silence=500)
        if chunks:
            for i, chunk in enumerate(chunks):
                if i > 99:
                    break
                chunk.export(r"{}\chunks\{}_{:02d}.wav".format(folder_name, filename[:-4], i), format="wav")
print('Script completed.')

Data generation

ForgeryPy3 is a fake data generator fully compatible with Python 2 and 3.

Python on Linux

Linux Mint 14 Nadia with Cinnamon

The basic install of Mint 14 with Cinnamon comes with Python 3.2.3 and 2.7.3.

Invoking Python 3.2 Interpreter

$ python3.2
Python 3.2.3 (default, Oct 19 2012, 19:53:16) 
[GCC 4.7.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>

Python stories

How others are using Python

Python at Netflix

Notes about Python 2

Avoid Python 2.x and use Python 3.x versions.

My computing notes