Documentation¶
Every example on this page is real: the output shown is exactly what the function returns.
Installation¶
Percentify requires Python 3.10+, numpy, and pandas 2.0+.
Conventions¶
A few rules hold across the whole library, so you always know what to expect:
- DataFrame in → DataFrame out. Pass a DataFrame and you get a clean DataFrame back. Pass a single
Series(where it makes sense) and you get a single number. - Sorted worst-first. Results come back ordered by what usually matters most: most collinear, most missing, most variable, most outliers.
decimalseverywhere. Every function accepts adecimalsargument (default2). Passdecimals=Nonefor full precision.- Numeric-only functions ignore text columns. They quietly skip non-numeric columns instead of crashing.
- Friendly warnings, not tracebacks. Hand a function something it can't use (e.g. an all-text DataFrame) and you get a
PercentifyWarningexplaining why, plus an empty result, never a raw NumPy/pandas stack trace. See Warnings.
Polars support¶
Every function accepts polars DataFrames and Series as well as pandas, and hands back the same kind you passed in: polars in, polars out. Detection is automatic from the input type, so there is nothing to configure.
import polars as pl
from percentify import missing
df = pl.DataFrame({"salary": [50000, None, 60000, None], "age": [25, 30, None, 40]})
missing(df) # returns a polars DataFrame
polars stays optional: it is only imported when you actually pass a polars object, so pandas-only users pay nothing. Conversion goes through Arrow, so pyarrow must be installed for the polars path (it usually already is in a polars setup).
profiler¶
The flagship, a one-call data diagnostician. pandas .describe() tells you what your data is; profiler() tells you what to do about it: every issue ranked worst-first, each with its fix. Instead of dumping statistics for every column, it runs a battery of checks, scores the data's health, and composes the rest of the toolkit (missing, imbalance, and more) under one entry point.
Similar concept
ydata-profiling / pandas.DataFrame.describe, but diagnostic (it ranks problems and suggests fixes) rather than descriptive.
Signature
Example
import pandas as pd
from percentify import profiler
df = pd.DataFrame({
"user_id": range(100), # identifier
"plan": ["free"] * 100, # constant
"age": [20 + (i * 37) % 40 for i in range(100)],
"spend": [None if i % 2 == 0 else float((i * 13) % 100) for i in range(100)],
"churn": ["No"] * 96 + ["Yes"] * 4, # imbalanced target
})
report = profiler(df, target="churn")
report.health # 87
report.to_frame()
severity code column message suggestion
warning constant plan only one distinct value drop, carries no information
warning high_missing spend 50% missing impute or drop
warning id_like user_id identifier-like (100/100 unique) further diagnostic required
info imbalance <target> class 'Yes' is only 4.0% of rows consider resampling or class weights
The report renders as a compact, color-coded summary in notebooks and terminals, and exposes everything programmatically:
report.errors,report.warnings,report.infos: findings filtered by severity.report.to_frame(): all findings as a tidy DataFrame.report.health: a 0 to 100 score (100 is clean).assert not report.errors: drop it straight into a CI data-quality gate.
Pass target= to also check for leakage (features that predict a target almost perfectly) and class imbalance. It accepts a single column name or a list of names for multi-target frames. Each target is set aside from the feature diagnostics and gets its own TARGET section showing its inferred role (classification or regression) and a balance / shape readout (the class balance for classification, the skew for regression), with the checks run against each. A numeric target that arrived as text (scattered values stored as strings) is coerced and read as regression, not mistaken for a giant class list. Accepts pandas or polars input.
profiler(df, target="price") # one target
profiler(df, target=["price", "sold"]) # two targets, each set aside and checked
Try it on your own data
Point profiler() at any messy dataset you have lying around, pandas or Polars, and read the findings top to bottom. It is the fastest way to see what to fix before you model.
change¶
Percentage change: as two numbers, between two columns, or down a whole series.
Similar concept
pandas.DataFrame.pct_change
Signature
Two numbers
Between two columns (element-wise), perfect for a new column:
import pandas as pd
df = pd.DataFrame({
"forecast": [100, 200, 50, 80],
"actual": [150, 150, 100, 80],
})
df["change_pct"] = change(df["forecast"], df["actual"])
df
Down one column (period-over-period):
The first value is NaN because there is no prior period to compare against. Passing a whole DataFrame applies this to every numeric column at once.
Note
Where old is 0, the result is 0.0 (safe division). Two Series are aligned by index, not position, which is fine when both columns come from the same DataFrame.
vif¶
Variance Inflation Factor: the classic multicollinearity check, without the six-line loop.
Similar concept
statsmodels.stats.outliers_influence.variance_inflation_factor
Signature
Example
import numpy as np, pandas as pd
from percentify import vif
np.random.seed(7)
base = np.random.randn(200)
df = pd.DataFrame({
"age": base + np.random.randn(200) * 0.1,
"income": base * 2 + np.random.randn(200) * 0.1, # closely tracks age
"score": np.random.randn(200),
})
vif(df)
age and income have sky-high VIFs (they carry the same signal), while score is independent.
Flag only the problem columns with flag:
Rules of thumb
VIF > 5 suggests moderate multicollinearity; VIF > 10 suggests severe. A perfectly collinear column returns inf.
missing¶
The percentage of missing values in each column, sorted highest first.
Similar concept
pandas.DataFrame.isna
Signature
Example
import pandas as pd
from percentify import missing
df = pd.DataFrame({
"salary": [50000, None, 60000, None],
"age": [25, 30, None, 40],
"city": ["NY", "LA", "SF", "LA"],
})
missing(df)
Unlike the numeric-only functions, missing reports on every column, text included.
cv¶
Coefficient of variation: relative variability (std ÷ mean), as a percentage. Handy for comparing spread across columns on different scales.
Similar concept
scipy.stats.variation
Signature
A single column returns a number
A DataFrame returns every numeric column, most variable first
df = pd.DataFrame({
"salary": [50000, 52000, 48000, 90000, 51000],
"age": [25, 30, 35, 40, 45],
})
cv(df)
outliers¶
The percentage of values that are outliers by the IQR method (below Q1 − 1.5·IQR or above Q3 + 1.5·IQR).
Similar concept
scipy.stats.iqr
Signature
A single column returns a number
import pandas as pd
from percentify import outliers
outliers(pd.Series([10, 11, 12, 13, 14, 15, 200]))
One value out of seven (200) is an outlier → 14.29%.
A DataFrame returns every numeric column
df = pd.DataFrame({
"salary": [10, 11, 12, 13, 14, 15, 200],
"age": [25, 26, 27, 28, 29, 30, 31],
})
outliers(df)
Tune the sensitivity with multiplier (e.g. multiplier=3.0 for a looser bound).
pca_variance¶
The percentage of variance explained by each principal component, plus a running cumulative total.
Similar concept
sklearn.decomposition.PCA (.explained_variance_ratio_)
Signature
Example
import numpy as np, pandas as pd
from percentify import pca_variance
np.random.seed(7)
base = np.random.randn(200)
df = pd.DataFrame({
"height": base + np.random.randn(200) * 0.3,
"weight": base + np.random.randn(200) * 0.3, # shares a signal with height
"noise": np.random.randn(200),
})
pca_variance(df)
Read the cumulative column to decide how many components to keep. Here, PC1 + PC2 already capture 97.4% of the variance.
Standardization matters
By default standardize=True, so each column is scaled to unit variance first. This stops a column measured in large units (e.g. dollars) from dominating purely because of its scale. Pass standardize=False for covariance-based PCA on the raw values.
pca_loadings¶
Principal component loadings: what each component is made of. Where pca_variance tells you how important each PC is, pca_loadings shows how much each original feature contributes to it, as a feature-by-component table.
Similar concept
sklearn.decomposition.PCA (.components_)
Signature
Example
import numpy as np, pandas as pd
from percentify import pca_loadings
np.random.seed(7)
base = np.random.randn(200)
df = pd.DataFrame({
"height": base + np.random.randn(200) * 0.3,
"weight": base + np.random.randn(200) * 0.3, # shares a signal with height
"noise": np.random.randn(200),
})
pca_loadings(df)
Read down a column: PC1 is height and weight together (both 0.71), while noise sits near 0, so PC1 is the shared height/weight signal. Standardized by default, matching pca_variance.
Sign is arbitrary
A component and its negation are equivalent, so the sign of a loading is not meaningful on its own. Only the relative signs and magnitudes within a column matter.
imbalance¶
Summarize class imbalance in a categorical target: the per-class breakdown plus the headline metrics you actually report.
Similar concept
pandas.Series.value_counts
Signature
Example
import pandas as pd
from percentify import imbalance
df = pd.DataFrame({"churn": ["No"] * 850 + ["Yes"] * 150})
result = imbalance(df["churn"])
result
The headline metrics come attached on .attrs["summary"], so the return stays a clean DataFrame:
{'n_classes': 2,
'majority_class': 'No',
'minority_class': 'Yes',
'imbalance_ratio': 5.67,
'entropy_pct': 60.98}
imbalance_ratio: the majority count divided by the minority count (5.67means "No" is 5.7x more common than "Yes").entropy_pct:100for a perfectly balanced target, approaching0as one class dominates.
Pass a single column (df["target"]), not the whole DataFrame. Nulls are dropped before counting.
correlate¶
Correlation with p-values, the piece df.corr() leaves out. Pass two Series for a single (r, p), or a whole DataFrame for a tidy table of every numeric pair, strongest first.
Similar concept
scipy.stats.pearsonr / scipy.stats.spearmanr
Signature
Two columns return (r, p)
import numpy as np, pandas as pd
from percentify import correlate
np.random.seed(7)
base = np.random.randn(200)
df = pd.DataFrame({
"age": base,
"income": base * 2 + np.random.randn(200) * 0.3, # tracks age
"score": np.random.randn(200),
})
correlate(df["age"], df["income"]) # (0.99, 0.0)
A DataFrame returns a ranked table
Pass method="spearman" for rank (monotonic) correlation.
skew_report¶
Distribution shape for every numeric column: skew, kurtosis, outlier percentage, and a suggested transform. Most-skewed first.
Similar concept
pandas.DataFrame.skew + pandas.DataFrame.kurt
Signature
Example
import numpy as np, pandas as pd
from percentify import skew_report
np.random.seed(1)
df = pd.DataFrame({
"income": np.random.exponential(2, 500), # right-skewed
"age": np.random.normal(40, 10, 500), # roughly symmetric
})
skew_report(df)
feature skew kurtosis outlier_pct suggested_transform
income 1.55 2.77 3.8 log1p
age -0.01 0.24 0.8 none
suggested_transform is a starting point: log1p for right-skewed non-negative data, yeo-johnson for skew with negatives, and none when a column is already roughly symmetric. The outlier_pct column reuses outliers.
bootstrap_ci¶
A confidence interval for any statistic, with no distribution assumed. It resamples the data with replacement and reads the percentiles of the resampled statistic.
Similar concept
scipy.stats.bootstrap
Signature
Example
import numpy as np
from percentify import bootstrap_ci
np.random.seed(1)
income = np.random.exponential(2, 500)
bootstrap_ci(income, random_state=0) # (1.88, 2.22)
Pass any statistic (for example np.median), change the level with ci, and set random_state for a reproducible interval.
permutation_test¶
A distribution-free p-value for the difference between two samples. It shuffles the group labels many times and asks how often chance alone produces a difference as large as the one you saw.
Similar concept
scipy.stats.permutation_test
Signature
Example
import numpy as np
from percentify import permutation_test
np.random.seed(1)
a = np.random.normal(100, 15, 120)
b = np.random.normal(106, 15, 120)
permutation_test(a, b, random_state=0) # 0.001
The default statistic is the difference in means; pass your own statistic(a, b) for anything else. It returns the number, not a pass or fail verdict, so the judgement stays with you.
effect_size¶
How big is the difference, not just whether it is significant. It detects the outcome type and reports the right measure.
Similar concept
pingouin.compute_effsize
Signature
Numeric outcome: Cohen's d, Hedges' g, mean difference
import numpy as np, pandas as pd
from percentify import effect_size
np.random.seed(1)
df = pd.DataFrame({
"variant": ["A"] * 500 + ["B"] * 500,
"revenue": np.concatenate([np.random.normal(100, 20, 500),
np.random.normal(112, 20, 500)]),
})
effect_size(df, group="variant", value="revenue")
Binary outcome (two levels): Cohen's h and lift
ab = pd.DataFrame({
"variant": ["A"] * 100 + ["B"] * 100,
"converted": [1] * 35 + [0] * 65 + [1] * 20 + [0] * 80,
})
effect_size(ab, group="variant", value="converted")
The interpretation column labels the magnitude (negligible / small / medium / large), so a raw number is never left without context.
difference¶
Symmetric percentage difference between two values or two columns: how far apart they are, regardless of direction. (Reach for change when direction matters.)
Similar concept
numpy / pandas element-wise arithmetic
Signature
Two numbers
Two columns (element-wise), using the average of the two values as the denominator, so difference(a, b) == difference(b, a):
import pandas as pd
sensors = pd.DataFrame({
"sensor_a": [10.0, 50.0, 100.0],
"sensor_b": [12.0, 50.0, 130.0],
})
sensors["pct_gap"] = difference(sensors["sensor_a"], sensors["sensor_b"])
sensors
split¶
Distribute a total across weights, proportionally: allocation for budgets, quotas, or apportioning a sum by group.
Similar concept
numpy / pandas weighted arithmetic
Signature
A list of weights returns a list
A column of weights returns an aligned Series, allocating a budget by population:
import pandas as pd
budget = pd.DataFrame({
"region": ["North", "South", "East"],
"population": [200, 300, 500],
})
budget["budget"] = split(10000, budget["population"])
budget
Raises ValueError if weights is empty or sums to zero.
display¶
Format a number, or a whole column, as clean percentage strings. The "last mile" for reports, dashboards, and exports.
Similar concept
pandas.Series.map + Python string formatting
Signature
A single number returns a string
A column returns a Series of strings, turning ratios into report-ready text:
import pandas as pd
rates = pd.DataFrame({
"campaign": ["A", "B", "C"],
"conv_rate": [0.045, 0.12, 0.083],
})
rates["display"] = display(rates["conv_rate"], multiply=True)
rates
Use multiply=True when your values are ratios (0.45 → "45.0%"); leave it off when they're already percentages (45 → "45.0%"). Customize the trailing text with suffix.
Warnings and non-numeric data¶
Percentify never hands you a raw traceback for a predictable mistake. Give a function data it can't use and it raises a PercentifyWarning and returns an empty result:
import pandas as pd
from percentify import vif
vif(pd.DataFrame({"city": ["NY", "LA"], "team": ["A", "B"]}))
PercentifyWarning: Numeric columns required: no numeric columns found. VIF
measures multicollinearity between numeric features - encode any
categorical/text columns first.
Leave them on while you learn; switch them off when you ship
Instead of a cryptic traceback, you get a friendly nudge about what percentify did and why, so one messy column never dents your momentum. In a trusted production pipeline, silence PercentifyWarning to keep your logs quiet.
The warning is a normal Python warning, so you can catch or silence it: