Merge pull request #3705 from KBVE/dev

Preparing Alpha Branch

apps/kbve.com/src/content/journal/01-09.mdx

@@ -20,6 +20,13 @@ tags:
 
     Going to finish up the helm chart and then we can move forward.
     The `pydiscordsh-main.yaml` is almost done, we just need to fix the ingress.
+    After finishing up the ingress and before the end of the night, I need to configure the sealed secrets.
+
+```shell
+
+        ./kbve.sh -nx kilobase:seal --namespace=discordsh --keyName=discord-turso --secrets=""
+
+```
 
 -   07:10PM
 
@@ -28,7 +35,25 @@ tags:
     Some quick hardcore afk game play!
     I am thinking that it will be a bit rough to do on the side, while still programming.
 
--   TODO - 09:00PM - Car battery should be charged.
+-   09:00PM
+
+    **Car**
+
+    Car battery should be charged!
+    Went out in the cold, got some greek yogurt and everything seems charged.
+
+-   11:52PM
+
+    **Deployment**
+
+    The deployment of `PyDiscordSh` is below:
+
+    Repo: `https://github.com/KBVE/kbve.git`
+    Branch : `dev`
+    Paths: `/migrations/kube/charts/discordsh`
+
+    There were some 503 errors but we were able to get everything wired!
+
 
 ## 2024
 
@@ -44,7 +69,8 @@ tags:
         I would like it so that the user has some integration settings but that might have to be done later on.
 
         Two additional libraries that I wanted to add were:
-        `reqwest`, at version 0.11 and `image`.        
+        `reqwest`, at version 0.11 and `image`.
+
 
 ## Quote
 

apps/kbve.com/src/content/journal/01-10.mdx

@@ -12,7 +12,30 @@ tags:
 
 ## Notes
 
-### 2024
+## 2025
+
+-   **Supabase**
+
+    01:15PM
+
+        The supabase integration with fastapi can be done in two ways, indirectly through the JWT or directly though a local connection to the supabase.
+        Before making the local connection to the supabase, I am going to do the remote connection first!
+        We just want to make sure that the user can login from the supabase instance.
+        Actually we also need to add the ingress to have a local connection but for now we can use a `https://supabase.kbve.com` during the testing phase.
+        In the future we want to use a `supabase.local` to handle it for us.
+
+    05:20PM
+
+        We were missing the PyJWT, so I am going to go ahead and add that into the repo.
+        To do this, we will use this nx command:
+
+```shell
+ pnpm nx run pydiscordsh:add --name pyjwt
+```
+
+        This will add the library to our `pydiscordsh`.
+
+## 2024
 
 -   11:04am - `WorkFlow`
 

apps/pydiscordsh/pydiscordsh/__init__.py

@@ -1,3 +1,3 @@
 from .api import Routes, CORS
 from .apps import TursoDatabase, DiscordServerManager
-from .api import SetupSchema, Hero, DiscordServer, Health, SchemaEngine
+from .api import SetupSchema, Hero, DiscordServer, Health, SchemaEngine, Utils

apps/pydiscordsh/pydiscordsh/api/__init__.py

@@ -1,4 +1,5 @@
 from .cors import CORS
 from .routes import Routes
 from .schema import SetupSchema, Hero, DiscordServer, SchemaEngine
-from .heath import Health
+from .heath import Health
+from .utils import Utils

apps/pydiscordsh/pydiscordsh/api/schema.py

@@ -1,18 +1,53 @@
 from typing import Optional, List
-import os, re
+import os, re, html
 from sqlmodel import Field, Session, SQLModel, create_engine, select, JSON, Column
-from pydantic import validator
+from pydantic import validator, root_validator
+import logging
+from pydiscordsh.api.utils import Utils
 
-class Hero(SQLModel, table=True):
+logger = logging.getLogger("uvicorn")
+
+class SanitizedBaseModel(SQLModel):
+    class Config:
+        arbitrary_types_allowed = True
+        validate_assignment = True
+
+    @staticmethod
+    def _sanitize_string(value: str, user_id: Optional[str] = None, server_id: Optional[int] = None) -> str:
+        sanitized = re.sub(r'[^a-zA-Z0-9\s.,;:!?-_://?=%()]', '', value)
+        sanitized = re.sub(r'<.*?>', '', sanitized)  # Strip HTML tags
+        if sanitized != value:
+            logging.error(f"Sanitization failed for value: '{value}'. Sanitized version: '{sanitized}'. Potential harmful content detected."
+                          f" User ID: {user_id}, Server ID: {server_id}")
+            raise ValueError("Invalid content in input: Contains potentially harmful characters.")
+        return html.escape(sanitized)
+
+    @root_validator(pre=True)
+    def sanitize_all_fields(cls, values):
+        user_id = values.get('user_id', None) #TODO: Pass user ID into this somwhow once we get users done
+        server_id = values.get('server_id', None)
+
+        for field, value in values.items():
+            if isinstance(value, str):
+                try:
+                    sanitized_value = cls._sanitize_string(value, user_id, server_id)
+                    values[field] = sanitized_value
+                except ValueError as e:
+                    logging.error(f"Failed to sanitize field '{field}' with value '{value}': {str(e)}"
+                                  f" User ID: {user_id}, Server ID: {server_id}")
+                    raise e
+        return values
+
+class Hero(SanitizedBaseModel, table=True):
     id: Optional[int] = Field(default=None, primary_key=True)
     name: str = Field(..., max_length=64)
     secret_name: str = Field(..., max_length=64)
     age: Optional[int] = Field(default=None, ge=0, le=10000)
 
-# class User(SQLModel, table=True):
+# class User(SanitizedBaseModel, table=True):
 #     user_id: int = Field(primary_key=True)
 
-class DiscordServer(SQLModel, table=True):
+class DiscordServer(SanitizedBaseModel, table=True):
     server_id: int = Field(primary_key=True)  # Pre-existing unique server ID
     owner_id: str = Field(nullable=False, max_length=50)
     lang: Optional[List[str]] = Field(default=None, sa_column=Column(JSON))
@@ -22,9 +57,9 @@ class DiscordServer(SQLModel, table=True):
     name: str = Field(..., max_length=100)
     summary: str = Field(..., max_length=255)
     description: Optional[str] = Field(default=None, max_length=1024)
-    website: str = Field(..., max_length=100)
+    website: Optional[str] = Field(default=None, max_length=255)
     logo: str = Field(..., max_length=255)
-    banner: str = Field(..., max_length=255)
+    banner: Optional[str] = Field(default=None, max_length=255)
     video: str = Field(..., max_length=255)
     bumps: int = Field(default=0, ge=0)  # Bumps or votes
     bump_at: Optional[int] = Field(default=None, nullable=True)  # UNIX timestamp for bump date
@@ -36,10 +71,17 @@ class DiscordServer(SQLModel, table=True):
     invoice_at: Optional[int] = Field(default=None, nullable=True)  # UNIX timestamp for the invoice date
     created_at: Optional[int] = Field(default=None, nullable=False)  # UNIX timestamp for creation date
     updated_at: Optional[int] = Field(default=None, nullable=True)  # UNIX timestamp for update date
-
-    class Config:
-        arbitrary_types_allowed = True
-        validate_assignment = True
+
+    @validator("website", "logo", "banner", pre=True, always=True)
+    def validate_common_urls(cls, value):
+        try:
+            return Utils.validate_url(value)
+        except ValueError as e:
+                # Log the error and raise a more specific error message
+                logger.error(f"URL validation failed for value: '{value}'")
+                raise ValueError(f"Invalid URL format for field '{cls.__name__}'. Please provide a valid URL.") from e
+        return value
+
     @validator("lang", pre=True, always=True)
     def validate_lang(cls, value):
         if value:
@@ -48,7 +90,7 @@ def validate_lang(cls, value):
             valid_languages = {"en", "es", "zh", "hi", "fr", "ar", "de", "ja", "ru", "pt", "it", "ko", "tr", "vi", "pl"}
             for lang in value:
                 if lang not in valid_languages:
-                    raise ValueError(f"Invalid language code: {lang}. Must be one of {', '.join(cls.valid_languages)}.")
+                    raise ValueError(f"Invalid language code: {lang}. Must be one of {', '.join(valid_languages)}.")
         return value
 
     @validator("invite", pre=True, always=True)
@@ -63,20 +105,6 @@ def validate_invite(cls, value):
         if re.match(plain_code_pattern, value):
             return value
         raise ValueError(f"Invalid invite link or invite code. Got: {value}")
-
-
-    @validator("website", pre=True, always=True)
-    def validate_website(cls, value):
-        if not value:
-            raise ValueError("Website must be a valid URL.")
-        value = value.strip()
-        if not value.startswith(("http://", "https://")):
-            value = "http://" + value
-        website_pattern = r"^(https?://(?:[a-zA-Z0-9-]+\.)+[a-zA-Z]{2,})(?:/[^\s]*)?$"
-        if not re.match(website_pattern, value):
-            raise ValueError(f"Invalid website URL: {value}")
-        return value
-
 
     @validator("categories", pre=True, always=True)
     def validate_categories(cls, value):
@@ -95,8 +123,7 @@ def validate_video(cls, value):
                 return value
         raise ValueError("Invalid YouTube video ID or URL.")
 
-# class BumpVote(SQLModel, table=False)
-
+# class BumpVote(SanitizedBaseModel, table=False)
 
 class SchemaEngine:
     def __init__(self):
@@ -117,7 +144,6 @@ def get_session(self) -> Session:
         """Provide the database session."""
         return Session(self.engine)
 
-
 class SetupSchema:
     def __init__(self, schema_engine: SchemaEngine):
         self.schema_engine = schema_engine
@@ -136,5 +162,4 @@ def fetch_hero_by_name(self, hero_name: str):
                 print(f"Hero found: {hero.name}")
             else:
                 print("Hero not found.")
-            return hero
-
+            return hero

apps/pydiscordsh/pydiscordsh/api/utils.py

@@ -0,0 +1,46 @@
+import re
+from urllib.parse import urlparse, unquote
+
+class Utils:
+    @staticmethod
+    def validate_url(value: str, allow_encoded: bool = False) -> str:
+        """
+        Validates and sanitizes a URL, rejecting any URLs with percent-encoded characters.
+
+        Args:
+            value (str): The input URL to validate.
+            allow_encoded (bool): If True, allow encoded characters; otherwise, decode them.
+
+        Returns:
+            str: The validated and sanitized URL.
+
+        Raises:
+            ValueError: If the URL is invalid or contains unsafe content.
+        """
+        if not value:
+            raise ValueError("URL cannot be empty.")
+        # If encoded characters are not allowed, ensure no '%' character is present
+        if '%' in value:
+            raise ValueError(f"URL contains percent-encoded characters, which are not allowed: {value}")
+        # Decode the URL if encoded characters are not allowed
+        if not allow_encoded:
+            value = unquote(value)
+        # Parse the URL
+        parsed = urlparse(value)
+        # Check for valid scheme and netloc
+        if not parsed.scheme or not parsed.netloc:
+            raise ValueError(f"Invalid URL structure: {value}")
+        # Ensure the scheme is HTTP or HTTPS
+        if parsed.scheme not in {"http", "https"}:
+            raise ValueError(f"URL must start with http:// or https://. Got: {value}")
+        # Check if the netloc (domain) is not just a single character or empty
+        if len(parsed.netloc) < 3 or '.' not in parsed.netloc:
+            raise ValueError(f"Invalid URL domain: {value}")
+        # Check if the netloc is just a scheme (like "https://")
+        if parsed.netloc == "":
+            raise ValueError(f"Invalid URL, missing domain: {value}")
+        # Additional sanitization (remove unsafe characters)
+        sanitized_url = re.sub(r'[^a-zA-Z0-9:/?&=_\-.,]', '', value)
+        if sanitized_url != value:
+            raise ValueError(f"URL contains unsafe characters: {value}")
+        return sanitized_url

apps/pydiscordsh/pyproject.toml

@@ -30,6 +30,7 @@ readme = 'README.md'
   sqlmodel = "^0.0.22"
   pydantic = "^2.10.4"
   sqlalchemy-libsql = "^0.1.0"
+  pyjwt = "^2.10.1"
 
   [tool.poetry.group.dev.dependencies]
   autopep8 = "2.3.1"

migrations/kube/charts/discordsh/discord/templates/pydiscordsh-main.yaml

@@ -10,12 +10,12 @@ spec:
   replicas: {{ .Values.pydiscordsh.replicaCount }}
   selector:
     matchLabels:
-      app: discordsh
+      app: pydiscordsh
       tier: pydiscordsh
   template:
     metadata:
       labels:
-        app: discordsh
+        app: pydiscordsh
         tier: pydiscordsh
     spec:
       containers:
@@ -29,6 +29,17 @@ spec:
             {{- end }}
           resources:
             {{- toYaml .Values.pydiscordsh.resources | nindent 12 }}
+          env:
+            - name: TURSO_AUTH_TOKEN
+              valueFrom:
+                secretKeyRef:
+                  name: discord-turso
+                  key: TURSO_AUTH_TOKEN
+            - name: TURSO_DATABASE_URL
+              valueFrom:
+                secretKeyRef:
+                  name: discord-turso
+                  key: TURSO_DATABASE_URL
 
 ---
 

migrations/kube/charts/discordsh/discord/templates/turso-sealed-secrets.yaml

@@ -0,0 +1,14 @@
+---
+apiVersion: bitnami.com/v1alpha1
+kind: SealedSecret
+metadata:
+  name: discord-turso
+  namespace: discordsh
+spec:
+  encryptedData:
+    TURSO_AUTH_TOKEN: AgBh1IY+fPAezWP1ibvObhf/fSuif1FN73MIs6LI9k1+lJAs1Abjnjx8Qxjb5YdgJ60RgO37kmKGVCrtiPT2Y5fYU4ydsR7mh2WOdOMxWLqKw39N4QrfVCAQX/1N/NszskkPLse40pv2xK+8YBIEOkoqDSq/qHM6UHndj4+Ogn0iJL7oSvERB86za83HvDHsU4CkHWNET8cPdFi3QlbeBRXmniKl9/Qtz2ZKyk8DPfJmOF3eAE4Y31Xyz/xkM4/JVaphCdxLDt58SIEBpSpSuiFMqvgkozX820qGlVR8uNAiJh5IaRguaOrpcUaKcvh1mGnLvIhKw2gRhBXVc+1glBaXab1FWehXkwO527IFuBfdAO1ARDAiMv3leuEwu2p4C0oVuGsKmBf+74WIj28NLw4T7TMjstdwkmiCHNlkbDBiz6i6cgdOQJy+ddlcwz117d0aorlwa7IfxaXt0JH/v5+5Mo2FCziZeH2P7eYzqvXeuBb+gunsUTFIAtI/Jjxvmv+EOH+bntjKeNQmiOFiSotnUTvCzTEcts4jH8QknvKGKUFqdN5q3QIGP5G/HKwe8D+4ShHQyIvV6kmtYt1pCmj0Pfm0kB/ji8AWR1I0zKsnW9Fj3LJm+C+uLByi0yy9oyiyPOjW60Og9FfO3PEUw7M9rZg1LUGem6sS6O2RPAFu97DvT3NzZW67hHXOLzk2qON71Y7tdyhTJaL0cwvBbJqZIDoTi453EHSyeZz27CkqyS5sbo4aShbz2zQRNubFsBxkX0RR7C1lJzXcFpRotHNRnUakuR9gJwJsKaAISHVKVzfF49arcK8/wYbQ7p7xlSnYRzF/HAnl3/9vrJLRsA6L2Cq3NGf20LDISeS1D//9cnFEX2MjmAl23i5a2iPQPheo6agGRU/An/kiRJgfpjcXhwq6qrT4bZYwqpqQL/cxjMW3DTmY4rW45WqRo/GSNyJg69S0rfigdQIco20xdH+P+aOK6unWx2mvfkk=
+    TURSO_DATABASE_URL: AgCeIXgWJMh367RfLyvnZMcJItaojsaOPoygUUu34ON5UZF9oWsxCKDjirhZ8medPRL9qSzLVuXpbN/ZHCrwwms1O/wqIuq/zBabI6WzKpiKC2qfN3jIlMiFk5S9nOvpsZSdWw05KK9GJddMeEmu4pkCpQeDEPvjR0N+sD6e1QDzGXrWR0uIDOSbvcnO1JaKyNHh65UKPSLhpuo6D8DxxrfjqmT3PGZDAiq65uUZtTqV7wTJjL0ryCZctZu23yHKjXigLQBWtOVSnLeYLAh9Tm6/schbTL0l5+5UdAaIdH7Rs3MsQ+SXgsz5Ouuxrjj9dBqMQS38PCOk7spEglAfhfsu/QWDqyCTjB1cuPzUOCHkRtaQ7QATeuCkurlwRqlwPaGXHKQpC38a65AowTTvqKHBHwqLoShUfiQDW8qt1ot4MZU0gUqzkkVaRvEWNLdps04b6EhxR34388Xfr3gqmEZmcx9SJZjWUHxvqhwPu2L1ExDWAVLpEivFqziJBU4ByXWjzqFGhR2Cz9gp1aT33Pth0ZbG7XgJvrU44kYbXfnP/xTRvq16uhH9xxWQpaiP0tOI6+U4Csf7UPM2vZk21Od9Na1RR0fWhM8yZYp/Bq1d8+NMxsfhg4uixnMOlkBzQjtdVlgKwiTmjoOy44m8oTRUyI6ijF72I6cT9quRyFrSiBodIGjDrTIadKwalUZdHfNizdGqbUpxpHrb1P6NrJ+v9pBHTodMcBz5ue226SXh
+  template:
+    metadata:
+      name: discord-turso
+      namespace: discordsh