w3resource

NumPy Input and Output: genfromtxt() function

numpy.genfromtxt() function

The genfromtxt() used to load data from a text file, with missing values handled as specified.

Each line past the first skip_header lines is split at the delimiter character, and characters following the comments character are discarded.

Syntax:

numpy.genfromtxt(fname, dtype=<class 'float'>, comments='#', delimiter=None, skip_header=0, skip_footer=0, converters=None,
missing_values=None, filling_values=None, usecols=None, names=None, excludelist=None, deletechars=None, replace_space='_', autostrip=False,
case_sensitive=True, defaultfmt='f%i', unpack=None, usemask=False, loose=True, invalid_raise=True, max_rows=None, encoding='bytes')

Version: 1.15.0

Parameter:

Name Description Required /
Optional
fname File, filename, list, or generator to read. If the filename extension is gz or bz2, the file is first decompressed.
Note that generators must return byte strings in Python 3k. The strings in a list or produced by a generator are treated as lines.
file, str, pathlib.Path, list of str, generator
Required
dtype Data type of the resulting array. If None, the dtypes will be determined by the contents of each column, individually.
dtype
Optional
comments The character used to indicate the start of a comment. All the characters occurring on a line after a comment are discarded
str
Optional
delimiter The string used to separate values. By default, any consecutive whitespaces act as delimiter.
An integer or sequence of integers can also be provided as width(s) of each field.
str, int, or sequence
Optional
skiprows skiprows was removed in numpy 1.10. Please use skip_header instead.
int
Optional
skip_header The number of lines to skip at the beginning of the file.
int
Optional
skip_footer The number of lines to skip at the end of the file.
int
Optional
converters The set of functions that convert the data of a column to a value.
The converters can also be used to provide a default value for missing data:
converters = {3: lambda s: float(s or 0)}.
variable
Optional
missing missing was removed in numpy 1.10. Please use missing_values instead.
variable
Optional
missing_values The set of strings corresponding to missing data.
variable
Optional
filling_values The set of values to be used as default when the data are missing.
variable
Optional
usecols Which columns to read, with 0 being the first. For example,
usecols = (1, 4, 5) will extract the 2nd, 5th and 6th columns.
sequence
Optional
names If names is True, the field names are read from the first valid line after the first skip_header lines.
If names is a sequence or a single-string of comma-separated names, the names will be used to define the field names in a structured dtype.
If names is None, the names of the dtype fields will be used, if any.
{None, True, str, sequence}
Optional
excludelist A list of names to exclude. This list is appended to the default list ['return','file','print'].
Excluded names are appended an underscore: for example, file would become file_.
sequence
Optional
deletechars A string combining invalid characters that must be deleted from the names.
str
Optional
defaultfmt A format used to define default field names, such as "f%i" or "f_%02i".
str
Optional
autostrip Whether to automatically strip white spaces from the variables.
bool
Optional
replace_space Character(s) used in replacement of white spaces in the variables names. By default, use a '_'.
char
Optional
case_sensitive If True, field names are case sensitive. If False or 'upper', field names are converted to upper case.
If 'lower', field names are converted to lower case.
{True, False, ‘upper’, ‘lower’}
Optional
unpack If True, the returned array is transposed, so that arguments may be unpacked using x, y, z = loadtxt(...)
bool
Optional
usemask If True, return a masked array. If False, return a regular array.
bool
Optional
loose If True, do not raise errors for invalid values.
bool
Optional
invalid_raise If True, an exception is raised if an inconsistency is detected in the number of columns.
If False, a warning is emitted and the offending lines are skipped.
bool
Optional
max_rows The maximum number of rows to read. Must not be used with skip_footer at the same time.
If given, the value must be at least 1. Default is to read the entire file.
int
Optional
encoding Encoding used to decode the inputfile. Does not apply when fname is a file object.
The special value 'bytes' enables backward compatibility workarounds that ensure that you receive
byte arrays when possible and passes latin1 encoded strings to converters.
Override this value to receive unicode arrays and pass strings as input to converters.
If set to None the system default is used. The default value is 'bytes'.
str
Optional

Returns: out : ndarray

Data read from the text file. If usemask is True, this is a masked array.

Notes:

  • When spaces are used as delimiters, or when no delimiter has been given as input, there should not be any missing data between two fields.
  • When the variables are named (either by a flexible dtype or with names, there must not be any header in the file (else a ValueError exception is raised).
  • Individual values are not stripped of spaces by default. When using a custom converter, make sure the function does remove spaces.

NumPy.genfromtxt() method Example-1:

Comma delimited file with mixed dtype

>>> import numpy as np
>>> from io import StringIO
>>> import numpy as np
>>> str = StringIO("1,1.5,pqrst")
>>> data1 = np.genfromtxt(str, dtype=[('myint','i8'),('myfloat','f8'),
... ('mystring','S5')], delimiter=",")
>>> data1

Output:

array((1, 1.5, b'pqrst'),
      dtype=[('myint', '<i8'), ('myfloat', '<f8'), ('mystring', 'S5')])

NumPy.genfromtxt() method Example-2:

Using dtype = None

>>> import numpy as np
>>> str.seek(0) # needed for StringIO example only
>>> data1 = np.genfromtxt(str, dtype=None, names = ['myint','myfloat','mystring'], delimiter=",")
>>> data1

Output:

array((1, 1.5, 'abcde'),
      dtype=[('myint', '<i8'), ('myfloat', '<f8'), ('mystring', '|S5')])

NumPy.genfromtxt() method Example-3:

Specifying dtype and names

>>> import numpy as np
>>> s.seek(0)
>>> data = np.genfromtxt(s, dtype="i8,f8,S5", names=['myint','myfloat','mystring'], delimiter=",")
>>> data

Output:

array((1, 1.5, b'pqrst'),
      dtype=[('myint', '<i4'), ('myfloat', '<f8'), ('mystring', 'S5')])

NumPy.genfromtxt() method Example-4:

An example with fixed-width columns

>>> import numpy as np
>>> str = StringIO("11.5pqrst")
>>> data1 = np.genfromtxt(str, dtype=None, names=['intvar','fltvar','strvar'], delimiter=[1,3,5])
>>> data1

Output:

array((1, 1.5, b'pqrst'),
      dtype=[('intvar', '<i4'), ('fltvar', '<f8'), ('strvar', 'S5')])

Python - NumPy Code Editor:

Previous: savetxt() function
Next: fromregex() function



Share this Tutorial / Exercise on : Facebook and Twitter