# pypdevsbbl.extra.mathutils module

This file contains the main functionality for having more control over boxplots.

These can be used for custom computation of boxplots, histograms and some other features, based upon certain algorithms. This way, it is possible to compute these values in situations where you don’t have access over the entire dataset, but only over a few values thereof.

There are also some additional helper functions that can be useful in other circumstances as well.

class `pypdevsbbl.extra.mathutils.``BoxplotAlgorithm`

  Bases: `enum.Enum`

  Helper enum for creating boxplots of series of \(N\) items.

  Boxplots are created with 5 values: [min, Q1, median, Q3, max]

  Even though Q1 and Q3 are also defined as being medians, some software packages use other algorithms for computing these values. This enum allows a quick and clean access of a set of these other algorithms.

  See also

  `linear_interpolation()`

  `MEDIAN` = 0

    Default computation where the Q1 and Q3 values are the medians of the lower list and the upper list of the set split on its global median, respectively.

    The global median is not included in the computation. When there is an even amount of items, the two values that make up the median are both excluded from the computation.

    Computes the element at indexes Q1 and Q3 and determines a linear interpolation on the series if the index is not an integer.

    \(Q1 = \begin{cases}\dfrac{N}{4}\textrm{ if }N\textrm{ is even}\\\\ \dfrac{N + 1}{4}\textrm{ if }N\textrm{ is odd}\end{cases}\qquad Q3 = \begin{cases}\dfrac{3\cdot N + 4}{4}\textrm{ if }N\textrm{ is even}\\\\ \dfrac{3\cdot N + 3}{4}\textrm{ if }N\textrm{ is odd}\end{cases}\)

  `TUKEY` = 1

    Computation that’s used by R in the `fivenum()` function and by `matplotlib` for plotting boxplots.

    The computation is the same as `MEDIAN`, except that the median is included in the computation.

    Computes the element at indexes Q1 and Q3 and determines a linear interpolation on the series if the index is not an integer.

    \(Q1 = \begin{cases}\dfrac{N + 2}{4}\textrm{ if }N\textrm{ is even}\\\\ \dfrac{N + 3}{4}\textrm{ if }N\textrm{ is odd}\end{cases}\qquad Q3 = \begin{cases}\dfrac{3\cdot N + 2}{4}\textrm{ if }N\textrm{ is even}\\\\ \dfrac{3\cdot N + 1}{4}\textrm{ if }N\textrm{ is odd}\end{cases}\)

  `MINITAB` = 2

    Computation that’s used by Minitab and by MS Excel (in any version later than 2007, in the `QUARTILE.EXC()` function).

    Computes the element at indexes Q1 and Q3 and determines a linear interpolation on the series if the index is not an integer.

    \(Q1 = \dfrac{N + 1}{4}\qquad Q3 = \dfrac{3\cdot N + 3}{4}\)

  `FREUND_PERLES` = 3

    Computation that’s used by MS Excel (version 2007 and earlier use `QUARTILE()`, which was replaced in later versions by `QUARTILE.INC()`), by R in the `summary()` function and by `numpy.percentile()` (with a linear interpolation).

    Computes the element at indexes Q1 and Q3 and determines a linear interpolation on the series if the index is not an integer.

    \(Q1 = \dfrac{N + 3}{4}\qquad Q3 = \dfrac{3\cdot N + 1}{4}\)

  `MOORE_MCCABE` = 4

    Similar to `MEDIAN`, as is it used in the TI-83 calculators for boxplots.

    Computes the element at indexes Q1 and Q3 and determines a linear interpolation on the series if the index is not an integer.

    \(Q1 = \begin{cases}\dfrac{N + 2}{4}\textrm{ if }N\textrm{ is even}\\\\ \dfrac{N + 1}{4}\textrm{ if }N\textrm{ is odd}\end{cases}\qquad Q3 = \begin{cases}\dfrac{3\cdot N + 2}{4}\textrm{ if }N\textrm{ is even}\\\\ \dfrac{3\cdot N + 3}{4}\textrm{ if }N\textrm{ is odd}\end{cases}\)

  `MENDENHALL_SINCICH` = 5

    Same as `MINITAB`, except this method uses rounding for its computations.

    Computes the element at indexes Q1 and Q3 and determines a linear interpolation on the series if the index is not an integer.

    \(Q1 = \left\lfloor\dfrac{N + 1}{4}\right\rfloor\qquad Q3 = \left\lceil\dfrac{3\cdot N + 3}{4}\right\rceil\)

  `FAME` = 6

    Computes the Q1 and Q3 values via the Fast Algorithm for Median Estimation.

    See also

    `FAME()`

  `P_SQUARE` = 7

    Computes the Q1 and Q3 values via the P^2 Algorithm for Dynamic Calculation of Quantiles and Histograms without Scoring Observations.

    See also

    - `histogram()`
    - `histostep()`
    - `PPP()`

`pypdevsbbl.extra.mathutils.``compute_boxplot`(series: list, algorithm: pypdevsbbl.extra.mathutils.BoxplotAlgorithm = <BoxplotAlgorithm.MEDIAN: 0>)

  Compute the boxplot values for a series.

  This function computes the [min, Q1, median, Q3, max] set of a series, based upon a certain algorithm.

  Note

  The median in the result is the definition-wise median and not computed with some other algorithm.

  See also

  `linear_interpolation()`

  Parameters:

  - series (list) – A list of numeric vales from which a boxplot must be computed.
  - algorithm (BoxplotAlgorithm) – The algorithm to use for computing the data. Even though the Q1 and Q3 values are technically defined as medians, some software actually use other computations for these values. Defaults to `BoxplotAlgorithm.MEDIAN`.

  Returns: | A dictionary with keys [min, Q1, median, Q3, max] and their corresponding values.

`pypdevsbbl.extra.mathutils.``FAME`(series: list, b=0.1)

  Do the FAME algorithm based upon a series.

  Loops throughout the entire series and tries to estimate the median. Please note that this is a mere estimate and not at all the actual median!

  FAME (Fast Algorithm for Median Estimation) is described in An Optimal Median Calculation Algorithm for Estimating Internet Link Delays From Active Measurements by Dima Feldman and Yuval Shavitt at the university of Tel-Aviv, 2007.

  See also

  - http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.108.7376&rep=rep1&type=pdf (paper)
  - http://www.eng.tau.ac.il/~shavitt/courses/LargeG/streaming-median.pdf

  Parameters:

  - series (list) – The series to loop through.
  - b (numeric) – Minimal initial step size. Defaults to 0.1.

  Returns: | The estimated median of the series.

`pypdevsbbl.extra.mathutils.``histogram`(series: list, b: int)

  Computes a b-cell histogram estimate of the series.

  Loops throughout the series and plots an estimate for a b+1 point histogram, using b equiprobable cells. When there are less then b observations, the series itself becomes the histogram.

  The source for this algorithm is based upon the P^2 Algorithm for Dynamic Calculation of Quantiles and Histograms without Scoring Observations by Raj Jain and Imrich Chlamtac at the Washington University in Saint Louis, 1985.

  See also

  - https://www.cse.wustl.edu/~jain/papers/ftp/psqr.pdf (paper)
  - `histostep()`
  - `PPP()`

  Parameters:

  - series (list) – The series of which to compute the histogram.
  - b (int) – The amount of cells to compute. Note that b+1 will identify the amount of ‘bars’ in the histogram.

  Returns: | An (x, y) tuple where x contains the x-locations of all values on the x-axis of the histogram and y contains all heights of all bars. Both x and y are lists of numbers of size b+1.

`pypdevsbbl.extra.mathutils.``histostep`(xj, b: int, q: list, n: list, N: int)

  Helper function for the histogram function.

  This function was extracted to make it easier for the algorithm to be used without knowledge of the entire series.

  See also

  - `histogram()`
  - `PPP()`

  Parameters:

  - xj (numeric) – The new item in the series.
  - b (int) – The amount of cells to compute.
  - q (list) – The datavalues of the bars in the histogram.
  - n (list) – The positions of the bars in the histogram.
  - N (int) – The current observation index.

`pypdevsbbl.extra.mathutils.``PPP`(q: list, n: list, i: int, d=1)

  Do a Piecewise-Parabolic Prediction.

  Assume we have a parabola passing through (n[i-1], q[i-1]), (n[i], q[i]) and (n[i+1], q[i+1]) which is of the form

  \[y = ax^2 + bx + c\]

  where (x, y) are coordinates in these points. The interpolated value (prediction) at location i can be determined via solving the equations set above.

  Based upon the PP-algorithm from https://www.cse.wustl.edu/~jain/papers/ftp/psqr.pdf

  Parameters:

  - q (list) – The heights of the points.
  - n (list) – The location of the points.
  - i (int) – The index where we will look.
  - d (int) – Either -1 (left) or 1 (right)

`pypdevsbbl.extra.mathutils.``linear_interpolation`(x, data: list)

  Compute the linear interpolation.

  Using the data, it will compute the value that should appear at index idx. This function will do a linear interpolation between the datapoints i-1 and i around x. (Whereas i-1=floor(x) and i=ceil(x).) We assume the line through these points follows the equation

  \[\begin{split}y &= mx + q \\ y - y1 &= (y2 - y1) / (x2 - x1) * (x - x1)\end{split}\]

  Parameters:

  - x (numeric) – The location to do the interpolation at. (the x-value)
  - data (list) – The list of y-values at integer values.

  Returns: | The value that corresponds with the idx in the list.

`pypdevsbbl.extra.mathutils.``sign`(num)

  Apply the sign function.

  Parameters: | num (numeric) – The value of which the sign needs to be determined.
  Returns: | \(\begin{cases}\begin{align}&1&\textrm{if num > 0}\\&-1&\textrm{if num < 0}\\&0&\textrm{otherwise }\end{align}\end{cases}\)

`pypdevsbbl.extra.mathutils.``binomial`(n, k)

  The binomial computation.

  The amount of possible ways to choose k out of n. This is the computation of the binomial coefficient C(n, k). Instead of the basic formula of `n! / (k! * (n - k)!)` where ‘x!’ stands for ‘x factorial’, this method will try and compute a space-efficient binomial (i.e. O(1)).

  It is based on the explanation on: https://www.geeksforgeeks.org/space-and-time-efficient-binomial-coefficient/

  Parameters:

  - n (numeric) – The total number (top number).
  - k (numeric) – The amount to choose (bottom number).

  Returns: | The binomial of (n k).

`pypdevsbbl.extra.mathutils.``binprob`(t, n, p)

  Compute the probability w.r.t. a binomial distribution.

  Assume X follows a discrete binomial distribution with size n and probability p: X ~ Bin(n, p)

  Parameters:

  - t (numeric) – The value to compute.
  - n (numeric) – The size of the distribution.
  - p (float) – The probability of the distribution.

  Returns: | Pr[X <= t]
  Return type: | The probability that X is less then t

`pypdevsbbl.extra.mathutils.``draw_boxplot`(minimum, Q1, median, Q3, maximum, outliers=False, labels=None)

  Get a boxplot-data object in matplotlib, based on its values.

  Using this function, it is possible to quickly draw a boxplot, based on its internal values. By default, matplotlib uses either bootstrapping or Gaussian-based asymptotic approximation (Tukey).

  This function therefore allows drawing boxplots based on other algorithms, but it does NOT draw the boxplots themselves. It yields a mere object, which can be used in the Axes.bpx method of matplotlib.

  Parameters:

  - minimum (numeric) – The smallest number to be drawn on the boxplot.
  - Q1 (numeric) – The lower quartile edge of the box.
  - median (numeric) – The median position indicator in the box.
  - Q3 (numeric) – The upper quartile edge of the box.
  - maximum (numeric) – The largest number to be drawn on the boxplot.
  - outliers (bool) – When true, this will make sure the whiskers are at most 1.5 * IQR away from the box. If the minimum and the maximum are outside of this range, they will be indicated as outliers. Do note that, as was the case with the compute_boxplot function, this method is a mere approximation of the boxplot, which becomes clear when comparing it to the actual boxplot. Theoretically, it’s a good approximation nonetheless. Defaults to `False`.
  - labels (iterable) – Labels for each dataset. See also cbook.boxplot_stats.labels

  Returns:

  A dictionary containing the results for each column of data. This is basically a transformed dictionary, based on:

      >>> cbook.boxplot_stats([minimum, Q1, median, Q3, maximum], labels=labels)[0]

  Example

      >>> import matplotlib.pyplot as plt
      >>>
      >>> fig, ax = plt.subplots()
      >>> boxplot = draw_boxplot(minimum, Q1, median, Q3, maximum)
      >>> ax.bpx(boxplot)       # draws the actual boxplot
      >>> plt.show()

`pypdevsbbl.extra.mathutils.``mean`(seq)

  Computes the mean of a sequence.

  Parameters: | seq (iterable) – The sequence to compute the mean for.
  Returns: | The mean of the sequence.

`pypdevsbbl.extra.mathutils.``var`(seq)

  Computes the variance of a sequence.

  Parameters: | seq (iterable) – The sequence to compute the variance for.
  Returns: | The variance of the sequence.

`pypdevsbbl.extra.mathutils.``std`(seq)

  Computes the standard derivation of a sequence.

  Parameters: | seq (iterable) – The sequence to compute the standard derivation for.
  Returns: | The standard derivation of the sequence.

`pypdevsbbl.extra.mathutils.``harmonic`(n: int, m=1)

  Computes the n-th generalized harmonic number.

  Parameters:

  - n (int) – The number to obtain.
  - m (numeric) – The power of each denominator. Defaults to 1, which also allows non-generalized numbers.

  Returns: | The n-th generalized harmonic number.

`pypdevsbbl.extra.mathutils.``harmonicinv`(y, m=1)

  Compute the inverse general harmonic number.

  I.e. \(GH(n, m) = y \iff IH(y, m) = n\)

  Parameters:

  - y (numeric) – The general harmonic number.
  - m (numeric) – The power to compute. Defaults to 1.

  Returns: | The inverse general harmonic number.