API Reference

module SLOPE

Sorted L-One Penalized Estimation

slope(x, y; kwargs...) -> SlopeFit

Fit a SLOPE (Sorted L1 Penalized Estimation) model to the provided data.

SLOPE is a regularization method that combines the L1 norm with a sorted penalty, encouraging both sparsity and grouping of features.

Arguments

• x: Matrix of predictors (dense or sparse)
• y: Response variable (vector)

Keyword Arguments

• α::Union{AbstractVector,Real,Nothing}=nothing: Alpha sequence for regularization path. If nothing, a sequence is automatically generated.
• λ::Union{AbstractVector,Symbol,Nothing}=:bh: Lambda (penalty weights) sequence. Can be a vector of custom weights, or a symbol specifying the sequence type:
  • :bh: Benjamini-Hochberg sequence
  • :gaussian: Gaussian sequence
  • :oscar: Octagonal Shrinkage and Clustering Algorithm for Regression
  • :lasso: Lasso (all weights equal to 1.0)
  If nothing, defaults to :bh.
• fit_intercept::Bool=true: Whether to fit an intercept term
• loss::Symbol=:quadratic: Loss function type. Options:
  • :quadratic: Gaussian (continuous response)
  • :logistic: Binomial (binary classification)
  • :multinomial: Multinomial (multi-class classification)
  • :poisson: Poisson (count data)
• centering::Symbol=:mean: Predictor centering method. Options: :mean (center by mean), :none (no centering)
• scaling::Symbol=:sd: Predictor scaling method. Options: :sd (scale by standard deviation), :none (no scaling)
• path_length::Int=100: Number of regularization path points
• tol::Float64=1e-5: Convergence tolerance for optimization
• max_it::Int=10000: Maximum number of iterations
• q::Float64=0.1: Parameter controlling the shape of the penalty weights sequence. Should be in the range (0, 1).
• max_clusters::Union{Int,Nothing}=nothing: Early path stopping criterion for maximum number of clusters (defaults to n+1)
• dev_change_tol::Float64=1e-5: Early path stopping criterion for tolerance of change in deviance
• dev_ratio_tol::Float64=0.999: Early path stopping criterion for tolerance of deviance ratio
• α_min_ratio::Union{Float64,Nothing}=nothing: Fraction of maximum α to use as minimum value in the regularization path. Defaults to 1e-2 if n > p * m, otherwise 1e-4.
• cd_type::Symbol=:permuted: Coordinate descent update type. Options: :permuted (random permutation order), :cyclic (sequential order)
• random_seed::Int=0: Random seed for reproducibility

Returns

A SlopeFit object.

Examples

using SLOPE

# Basic regression example
x = randn(100, 20)
y = x[:, 1:5] * ones(5) + randn(100)
fit = slope(x, y)

# Use custom regularization parameters
fit = slope(x, y, q=0.05, path_length=50)

# Use OSCAR-type lambda sequence
fit = slope(x, y, λ=:oscar)

# Logistic regression
x = randn(100, 10)
y = rand([0, 1], 100)
fit = slope(x, y, loss=:logistic)

# Poisson regression (count data)
x = randn(100, 10)
y = rand(0:10, 100)
fit = slope(x, y, loss=:poisson)

# Multinomial classification
x = randn(150, 10)
y = repeat([1, 2, 3], 50)
fit = slope(x, y, loss=:multinomial)

SlopeFit

A structure containing the results of fitting a SLOPE model.

Fields

• intercepts::Vector{Vector{Float64}}: A vector of intercept vectors along the regularization path. For each point in the path, contains a vector of length m with class-specific intercepts.
• coefficients::Vector{SparseMatrixCSC{Float64,Int}}: A vector of sparse coefficient matrices along the regularization path. Each matrix is of size p×m where p is the number of predictors and m is the number of response classes (1 for regression).
• α::Vector{Float64}: The alpha values used at each point of the regularization path.
• λ::Vector{Float64}: The lambda values used at each point of the regularization path.
• m::Int: The number of response classes (1 for regression, >1 for multinomial).
• loss::Symbol: The loss function used in the model fitting process.
• classes::Union{Vector,Nothing}: A vector of unique class labels for the response variable. This is nothing for regression models (continuous responses).

Examples

using SLOPE

# Fit a SLOPE model
x = randn(100, 10)
y = randn(100)
fit = slope(x, y)

# Access results
fit.coefficients[1]  # Coefficients at first point of regularization path
fit.α  # Alpha values along the path
fit.intercepts[end]  # Intercept at last point of regularization path

predict(fit::SlopeFit, x) -> Vector{Matrix{Float64}}

Generate predictions from a fitted SLOPE model.

Returns predictions for each point along the regularization path. For regression models, predictions are continuous values. For classification models (logistic, multinomial), predictions are class probabilities or predicted class labels depending on the loss function.

Arguments

• fit::SlopeFit: A fitted SLOPE model from slope
• x::Union{AbstractMatrix,SparseMatrixCSC}: Predictor matrix (n × p)

Returns

A vector of prediction matrices, one for each point in the regularization path. Each matrix has dimensions (n × m), where n is the number of observations and m depends on the model type:

• For regression (:quadratic): m = 1 (predicted values)
• For Poisson (:poisson): m = 1 (predicted counts/rates)
• For logistic (:logistic): m = 1 (predicted probabilities)
• For multinomial (:multinomial): m = number of classes (class probabilities)

Examples

using SLOPE

# Regression
x = randn(100, 10)
y = randn(100)
fit = slope(x, y)
x_new = randn(20, 10)
predictions = predict(fit, x_new)
predictions[1]  # Predictions at first regularization point

# Classification
x = randn(100, 10)
y = rand([0, 1], 100)
fit = slope(x, y, loss=:logistic)
predictions = predict(fit, x_new)
predictions[end]  # Predictions at last regularization point

coef(fit::SlopeFit; index=nothing, α=nothing, simplify=false, refit=false)

Extract coefficients from a fitted SLOPE model.

Arguments

• fit::SlopeFit: A fitted SLOPE model object
• index::Union{Int, Nothing}=nothing: Index of the regularization path point to extract. If nothing (default), returns coefficients for all path points.
• α::Union{Real, Nothing}=nothing: Specific alpha value to extract coefficients for. If the alpha value is not in the fitted path, coefficients are interpolated by default. Cannot be used together with index.
• simplify::Bool=false: Whether to simplify the output format:
  • For single response models (m=1): Returns a matrix or vector instead of sparse matrices
  • When index=nothing and simplify=true: Returns a 3D array (p×m×pathlength) or 2D matrix (p×pathlength) if m=1
  • When index is specified and simplify=true: Returns a vector if m=1, otherwise a matrix
• refit::Bool=false: If true and α is provided, refit the model at the exact alpha value instead of interpolating. Requires that the original data is available (currently not implemented).

Returns

• If index=nothing and α=nothing and simplify=false: Vector of sparse matrices, one per path point
• If index=nothing and simplify=true: 3D array (p×m×path_length), or 2D if m=1
• If index or α is specified and simplify=false: Sparse matrix of size p×m
• If index or α is specified and simplify=true: Matrix of size p×m, or vector if m=1

Examples

using SLOPE

# Fit a SLOPE model
x = randn(100, 20)
y = x[:, 1:5] * ones(5) + randn(100)
fit = slope(x, y)

# Get all coefficients along the path (as sparse matrices)
all_coefs = coef(fit)

# Get coefficients at a specific path index
coef_10 = coef(fit, index=10)

# Get coefficients at a specific alpha value (interpolated)
coef_alpha = coef(fit, α=0.5)

# Get coefficients as a matrix (for single response)
coef_matrix = coef(fit, simplify=true)  # Returns p×path_length matrix

# Get coefficients at a specific alpha as a vector (for single response)
coef_vec = coef(fit, α=0.5, simplify=true)

# Multinomial example
x = randn(150, 10)
y = repeat([1, 2, 3], 50)
fit_multi = slope(x, y, loss=:multinomial)
coef_array = coef(fit_multi, simplify=true)  # Returns p×m×path_length array

slopecv(
  x,
  y;
  α=nothing,
  λ=:bh,
  γ=[0.0],
  q=[0.1],
  n_folds=10,
  n_repeats=1,
  metric=:mse,
  kwargs...
)

Perform cross-validation for SLOPE to find optimal hyperparameters.

Arguments

• x::Union{AbstractMatrix,SparseMatrixCSC}: Input feature matrix, can be dense or sparse.
• y::AbstractVector: Response vector.

Keyword Arguments

• α::Union{AbstractVector,Real,Nothing}=nothing: SLOPE regularization path. If nothing, it's automatically generated.
• λ::Union{AbstractVector,Symbol,Nothing}=:bh: Sequence of regularization parameters. If nothing, it uses the default BH sequence (:bh).
• γ::Union{AbstractVector,Real}=[0.0]: Parameter controlling the regularization sequence. Multiple values create a grid search.
• q::Union{AbstractVector}=[0.1]: FDR parameter for BH sequence. Multiple values create a grid search.
• n_folds::Int=10: Number of cross-validation folds.
• n_repeats::Int=1: Number of times to repeat the CV process with different fold assignments.
• metric::Symbol=:mse: Evaluation metric for cross-validation. Options include :mse, :mae, :accuracy, etc.
• kwargs...: Additional parameters passed to the SLOPE solver.

Returns

A SlopeCvResult object.

Examples

# Basic usage with default parameters
result = slopecv(X, y)

# Cross-validation with custom parameters
result = slopecv(X, y, γ=[0.0, 0.1, 0.5], q=[0.1, 0.05], n_folds=5, metric=:accuracy)

# Access best parameters and score
best_q = result.best_params["q"]
best_γ = result.best_params["γ"]
best_score = result.best_score

See Also

• slope: For fitting a SLOPE model with fixed parameters.

SlopeGridResult

Results for a specific hyperparameter combination in the SLOPE cross-validation grid search.

Fields

• params::Dict{String,Any}: Dictionary of hyperparameter values (e.g., "q", "γ")
• scores::Matrix{Real}: Cross-validation scores for each fold and alpha value
• alphas::Vector{Real}: Sequence of alpha values for the regularization path
• scores_means::Vector{Real}: Mean score across folds for each alpha
• scores_errors::Vector{Real}: Standard errors of scores across folds

SlopeCvResult

Result structure from SLOPE cross-validation.

Fields

• metric::Symbol: The evaluation metric used (e.g., :mse, :accuracy)
• best_score::Real: The best score achieved during cross-validation
• best_ind::Int: Index of the best parameter combination
• best_α_ind::Int: Index of the best alpha value in the regularization path
• best_params::Dict{String,Any}: Dictionary with the best parameter values
• results::Vector{SlopeGridResult}: Grid search results, of type SlopeGridResult for each parameter combination

plot(cvresult::SlopeCvResult; xvar=:α, index=1; kwargs...)

Plots the cross-validation results from a cross-validated SLOPE model.

Arguments

• cvresult::SLOPE.SlopeCvResult: The result of a cross-validated SLOPE model, containing multiple cross-validation results for different parameters.
• xvar::Symbol=:α: Variable for the x-axis, options:
  • :α: Plot against the regularization parameter alpha (default)
  • :step: Plot against the step number in the regularization path
• index::Int=1: Index of the cross-validation result to plot. If there are multiple values of γ or q, this specifies which set of these to visualize.

Keyword Arguments

• kwargs...: Additional arguments passed to the plot, such as:
  • title: Title for the plot
  • legend: Legend position (default: :none)
  • lw: Line width
  • color: Color scheme

Returns

A plot object showing the cross-validation error, with ribbons for standard error.

Examples

using SLOPE, Plots

# Fit a cross-validated SLOPE model
x = randn(100, 20)
y = x[:, 1:5] * ones(5) + randn(100)
cvresult = slopecv(x, y, n_folds=5)

# Plot cross-validation results
plot(cvresult)

plot(fit::SLOPE.SlopeFit; xvar=:α, response=1, kwargs...)

Plot the coefficient paths from a SLOPE model regularization path.

This function visualizes how the coefficients change along the regularization path, allowing you to see which variables enter the model and how their effects change as the regularization strength varies.

Arguments

• fit::SLOPE.SlopeFit: A fitted SLOPE model object containing the regularization path
• xvar::Symbol=:α: Variable for the x-axis, options:
  • :α: Plot against the regularization parameter alpha (default)
  • :step: Plot against the step number in the regularization path
• response::Int=1: For multi-response models, specifies which response's coefficients to plot
• layout: (DEPRECATED) Layout for multi-class plots, e.g., (rows, cols). Default is (m, 1) for m classes.

Keyword Arguments

• kwargs...: Additional arguments passed to the plot, such as:
  • title: Title for the plot
  • legend: Legend position (default: :none)
  • lw: Line width
  • color: Color scheme

Returns

A plot object showing the coefficient paths

Examples

using SLOPE
using Plots

# Fit a SLOPE model
x = randn(100, 20)
y = x[:, 1:5] * ones(5) + randn(100)
fit = slope(x, y)

# Plot coefficient paths against alpha
plot(fit)

# Plot against step number instead
plot(fit, xvar=:step)

# Customize the plot
plot(fit, title="SLOPE Coefficient Paths", lw=2)

regweights(
  p::Int;
  q::Float64=0.1,
  type::Symbol=:bh,
  n=nothing,
  θ1::Real=1.0,
  θ2::Real=1.0
)

Generates a sequence of regularization weights for the sorted L1 norm.

Arguments

• p::Int: The number of lambda values to generate (number of features)

Keyword Arguments

• q::Float64=0.1: The false discovery rate (FDR) level or quantile value (must be in (0, 1))
• type::Symbol=:bh: The type of sequence to generate:
  • :bh: Benjamini-Hochberg sequence
  • :gaussian: Gaussian sequence
  • :oscar: Octagonal Shrinkage and Clustering Algorithm for Regression
  • :lasso: Lasso (all weights equal to 1.0)
• n::Union{Int, Nothing}=nothing: Number of observations (required for :gaussian type)
• θ1::Real=1.0: First parameter for OSCAR weights
• θ2::Real=1.0: Second parameter for OSCAR weights

Returns

• Vector{Float64}: The generated lambda sequence in decreasing order

Examples

λ = regweights(100)  # Benjamini-Hochberg with default q=0.1
λ = regweights(100, q=0.05, type=:gaussian, n=200)
λ = regweights(50, type=:oscar, θ1=1.5, θ2=0.5)