[Bug]: Failed to get ssl certificate because it was not serialized correctly

[wakaka6]

crawl4ai version

0.4.3b3

Expected Behavior

API return ssl certification

Current Behavior

client:
Internal Server Error

server:
"POST /crawl_direct HTTP/1.1" 500 Internal Server Error

Is this reproducible?

Yes

Inputs Causing the Bug

curl -X 'POST' \
  'http://192.168.1.2:11235/crawl_direct' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
  "urls": "https://example.com",
  "magic": true,
  "extra": {
      "page_timeout": 30000,
      "proxy": "",
      "scan_full_page": true,
      "fetch_ssl_certificate": true,
      "user_agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:92.0) Gecko/20100101 Firefox/92.0"
    },
  "cache_mode": "disabled"
}'

Steps to Reproduce

Code snippets

OS

Linux

Python version

3.12

Browser

No response

Browser version

No response

Error logs & Screenshots (if applicable)

INFO:     192.168.1.2:56052 - "POST /crawl_direct HTTP/1.1" 500 Internal Server Error
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "~/mambaforge/envs/crawl4ai/lib/python3.12/site-packages/uvicorn/protocols/http/h11_impl.py", line 403, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "~/mambaforge/envs/crawl4ai/lib/python3.12/site-packages/uvicorn/middleware/proxy_headers.py", line 60, in __call__
    return await self.app(scope, receive, send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "~/mambaforge/envs/crawl4ai/lib/python3.12/site-packages/fastapi/applications.py", line 1054, in __call__
    await super().__call__(scope, receive, send)
  File "~/mambaforge/envs/crawl4ai/lib/python3.12/site-packages/starlette/applications.py", line 112, in __call__
    await self.middleware_stack(scope, receive, send)
  File "~/mambaforge/envs/crawl4ai/lib/python3.12/site-packages/starlette/middleware/errors.py", line 187, in __call__
    raise exc
  File "~/mambaforge/envs/crawl4ai/lib/python3.12/site-packages/starlette/middleware/errors.py", line 165, in __call__
    await self.app(scope, receive, _send)
  File "~/mambaforge/envs/crawl4ai/lib/python3.12/site-packages/starlette/middleware/cors.py", line 85, in __call__
    await self.app(scope, receive, send)
  File "~/mambaforge/envs/crawl4ai/lib/python3.12/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "~/mambaforge/envs/crawl4ai/lib/python3.12/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    raise exc
  File "~/mambaforge/envs/crawl4ai/lib/python3.12/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
    await app(scope, receive, sender)
  File "~/mambaforge/envs/crawl4ai/lib/python3.12/site-packages/starlette/routing.py", line 715, in __call__
    await self.middleware_stack(scope, receive, send)
  File "~/mambaforge/envs/crawl4ai/lib/python3.12/site-packages/starlette/routing.py", line 735, in app
    await route.handle(scope, receive, send)
  File "~/mambaforge/envs/crawl4ai/lib/python3.12/site-packages/starlette/routing.py", line 288, in handle
    await self.app(scope, receive, send)
  File "~/mambaforge/envs/crawl4ai/lib/python3.12/site-packages/starlette/routing.py", line 76, in app
    await wrap_app_handling_exceptions(app, request)(scope, receive, send)
  File "~/mambaforge/envs/crawl4ai/lib/python3.12/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    raise exc
  File "~/mambaforge/envs/crawl4ai/lib/python3.12/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
    await app(scope, receive, sender)
  File "~/mambaforge/envs/crawl4ai/lib/python3.12/site-packages/starlette/routing.py", line 73, in app
    response = await f(request)
               ^^^^^^^^^^^^^^^^
  File "~/mambaforge/envs/crawl4ai/lib/python3.12/site-packages/fastapi/routing.py", line 327, in app
    content = await serialize_response(
              ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "~/mambaforge/envs/crawl4ai/lib/python3.12/site-packages/fastapi/routing.py", line 181, in serialize_response
    return field.serialize(
           ^^^^^^^^^^^^^^^^
  File "~/mambaforge/envs/crawl4ai/lib/python3.12/site-packages/fastapi/_compat.py", line 151, in serialize
    return self._type_adapter.dump_python(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "~/mambaforge/envs/crawl4ai/lib/python3.12/site-packages/pydantic/type_adapter.py", line 527, in dump_python
    return self.serializer.to_python(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
pydantic_core._pydantic_core.PydanticSerializationError: Unable to serialize unknown type: <class 'crawl4ai.ssl_certificate.SSLCertificate'>

[wakaka6]

@aravindkarnam Older versions of the server code main.py used result.dict() as the api return, but crawl4ai.ssl_certificate.SSLCertificate is an object, so there is no way to serialize it. It seems that this code has been removed in the new version.