{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Custom Objective Tutorial\n",
    "\n",
    "Perpetual allows you to define your own objective functions for optimization. This is useful when the standard loss functions (like SquaredLoss or LogLoss) do not fit your specific use case.\n",
    "\n",
    "In this tutorial, we will demonstrate how to define a custom objective and use it with `PerpetualBooster`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Imports and Data Preparation\n",
    "\n",
    "We'll use the California Housing dataset for this regression example."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "from perpetual import PerpetualBooster\n",
    "from sklearn.datasets import fetch_california_housing\n",
    "from sklearn.metrics import mean_squared_error\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "# Load data\n",
    "data = fetch_california_housing()\n",
    "X = pd.DataFrame(data.data, columns=data.feature_names)\n",
    "y = data.target\n",
    "X_train, X_test, y_train, y_test = train_test_split(\n",
    "    X, y, test_size=0.2, random_state=42\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Defining a Custom Objective\n",
    "\n",
    "A custom objective in Perpetual is a tuple of three functions:\n",
    "\n",
    "1.  **loss(y, pred, weight, group)**: Computes the loss for each sample.\n",
    "2.  **gradient(y, pred, weight, group)**: Computes the gradient and hessian for each sample. It should return a tuple `(gradient, hessian)`. If `hessian` is constant (e.g., 1.0 for SquaredLoss), return `None` to improve performance.\n",
    "3.  **initial_value(y, weight, group)**: Computes the initial prediction for the model (usually the mean or median of the target).\n",
    "\n",
    "Let's implement a simple Squared Error objective as an example."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def custom_loss(y, pred, weight, group):\n",
    "    return (y - pred) ** 2\n",
    "\n",
    "\n",
    "def custom_gradient(y, pred, weight, group):\n",
    "    # Gradient of (y - pred)^2 with respect to pred is 2 * (pred - y)\n",
    "    # Note: Perpetual handles the scaling, so (pred - y) is sufficient.\n",
    "    grad = pred - y\n",
    "    hess = None  # If hess is constant (e.g. 1.0) return None to improve performance\n",
    "    return grad, hess\n",
    "\n",
    "\n",
    "def custom_init(y, weight, group):\n",
    "    return np.mean(y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Training and Evaluation\n",
    "\n",
    "Now we can pass our custom objective to the `PerpetualBooster` constructor."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Initialize booster with custom objective\n",
    "model = PerpetualBooster(objective=(custom_loss, custom_gradient, custom_init))\n",
    "\n",
    "# Fit the model\n",
    "model.fit(X_train, y_train)\n",
    "\n",
    "# Make predictions\n",
    "preds = model.predict(X_test)\n",
    "\n",
    "# Evaluate\n",
    "mse = mean_squared_error(y_test, preds)\n",
    "print(f\"Mean Squared Error with Custom Objective: {mse:.4f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. Comparison with Standard Objective\n",
    "\n",
    "Let's compare this with the built-in `SquaredLoss` objective."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "standard_model = PerpetualBooster(objective=\"SquaredLoss\")\n",
    "standard_model.fit(X_train, y_train)\n",
    "standard_preds = standard_model.predict(X_test)\n",
    "\n",
    "standard_mse = mean_squared_error(y_test, standard_preds)\n",
    "print(f\"Mean Squared Error with Standard Objective: {standard_mse:.4f}\")\n",
    "\n",
    "assert np.allclose(preds, standard_preds, atol=1e-5)\n",
    "print(\"Results are identical!\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}