Promatranje AI agenata
AI crawleri — GPTBot, ClaudeBot, PerplexityBot i drugi — svakodnevno čitaju vašu web-stranicu, a većina vlasnika to nikada ne vidi. Način promatranja razina je 1 SilentShieldove kontrole AI agenata: bilježi koje AI usluge posjećuju koje stranice — bez blokiranja i bez promjene ponašanja vaše web-stranice.
Vidljivost dolazi prije kontrole: tek kada znate tko čita vaš sadržaj, možete razumno odlučiti koga ćete dopustiti, ograničiti ili blokirati. Svako se viđenje stoga razvrstava u jednu od tri kategorije:
Ovo samo promatra — nikad ne blokira
Isječci koda na ovoj stranici rade po načelu 'postavi i zaboravi': izvještavaju koji AI agenti posjećuju stranicu i nikada ne mijenjaju odgovor. Postavljanje pravila na "Blokiraj" na nadzornoj ploči ne stupa na snagu samo iz ovog isječka. Za stvarno blokiranje (403 za nedopuštene botove) potrebna je zasebna enforce komponenta — razina 2 — koja se izvodi na vašem poslužitelju.
- Treniranje
- Crawler prikuplja vaš sadržaj za treniranje budućih AI modela (npr. GPTBot, ClaudeBot, CCBot).
- AI pretraživanje
- Usluga čita stranice kako bi odgovorila na pitanja korisnika i pritom može citirati vašu web-stranicu kao izvor (npr. OAI-SearchBot, PerplexityBot).
- Agenti
- Bot djeluje u stvarnom vremenu u ime konkretne osobe i primjerice istražuje proizvod ili ispunjava obrazac na vašoj web-stranici (npr. ChatGPT-User, Claude-User).
Postavljanje po platformi
WordPress
Ugrađeni observer stiže s nadolazećim ažuriranjem SilentShield dodatka. Do tada aktivirajte praćenje malim must-use dodatkom — jedna datoteka, bez konfiguracije:
Stvorite datoteku wp-content/mu-plugins/silentshield-observer.php s ovim sadržajem (mu-dodaci se učitavaju automatski; izbrišite datoteku za deaktivaciju):
<?php
/**
* Plugin Name: SilentShield — AI Agent Observer
* Description: Reports AI crawler visits to SilentShield (fire-and-forget;
* IP/UA are hashed server-side, GDPR). Delete this file to disable.
*
* Install: save this file as wp-content/mu-plugins/silentshield-observer.php
* (mu-plugins load automatically — no activation needed). To force it off
* without deleting: define('SILENTSHIELD_OBSERVER', false) in wp-config.php.
* An upcoming SilentShield plugin update will build this in.
*/
if (!defined('ABSPATH')) exit;
if (defined('SILENTSHIELD_OBSERVER') && !SILENTSHIELD_OBSERVER) return;
const SS_OBSERVE_URL = "https://api.silentshield.io/api/v1/agent/telemetry";
const SS_API_KEY = "YOUR_API_KEY"; // publishable site key
// For the always-fresh list, cache https://api.silentshield.io/api/v1/agent/bot-directory daily.
$ss_bots = ["GPTBot","OAI-SearchBot","ChatGPT-User","ClaudeBot","Claude-User",
"PerplexityBot","Perplexity-User","Bingbot","Applebot","Amazonbot","CCBot",
"Bytespider","meta-externalagent","Google-Agent"];
register_shutdown_function(function () use ($ss_bots) {
$ua = $_SERVER["HTTP_USER_AGENT"] ?? "";
$signed = !empty($_SERVER["HTTP_SIGNATURE"]);
$isBot = $signed;
foreach ($ss_bots as $b) { if (stripos($ua, $b) !== false) { $isBot = true; break; } }
if (!$isBot) return;
if (function_exists("fastcgi_finish_request")) { fastcgi_finish_request(); }
wp_remote_post(SS_OBSERVE_URL, [
"timeout" => 3,
"blocking" => false,
"headers" => ["Content-Type" => "application/json", "x-api-key" => SS_API_KEY],
"body" => wp_json_encode(["sightings" => [[
"ua" => $ua,
"ip" => $_SERVER["REMOTE_ADDR"] ?? "",
"path" => strtok($_SERVER["REQUEST_URI"] ?? "/", "?"),
"method" => $_SERVER["REQUEST_METHOD"] ?? "GET",
]]]),
]);
});Next.js
Samostalan middleware — bez SDK-a. Praćenje se izvodi putem event.waitUntil(), izvan puta odgovora, pa nikada ne usporava vaše stranice:
// middleware.ts — self-contained observer, no SDK required.
// Sightings are reported via event.waitUntil(), off the response path,
// so your pages never wait. Best-effort and fail-open by design.
import { NextResponse, type NextRequest } from "next/server";
const OBSERVE_URL = "https://api.silentshield.io/api/v1/agent/telemetry";
const API_KEY = "YOUR_API_KEY"; // publishable site key
// Known AI-agent UA tokens. For the always-fresh list, fetch
// https://api.silentshield.io/api/v1/agent/bot-directory once a day and cache it.
const BOTS = ["GPTBot", "OAI-SearchBot", "ChatGPT-User", "ClaudeBot", "Claude-User",
"PerplexityBot", "Perplexity-User", "Bingbot", "Applebot", "Amazonbot", "CCBot",
"Bytespider", "meta-externalagent", "Google-Agent"];
export function middleware(req: NextRequest, event: { waitUntil(p: Promise<unknown>): void }) {
const ua = req.headers.get("user-agent") ?? "";
const signed = !!req.headers.get("signature");
if (signed || BOTS.some((b) => ua.toLowerCase().includes(b.toLowerCase()))) {
const sighting = {
ua,
path: req.nextUrl.pathname,
method: req.method,
ip: req.headers.get("x-forwarded-for")?.split(",")[0]?.trim(),
...(signed && {
signature: req.headers.get("signature"),
signature_input: req.headers.get("signature-input"),
signature_agent: req.headers.get("signature-agent"),
authority: req.nextUrl.host,
scheme: req.nextUrl.protocol.replace(":", ""),
}),
};
event.waitUntil(
fetch(OBSERVE_URL, {
method: "POST",
headers: { "Content-Type": "application/json", "x-api-key": API_KEY },
body: JSON.stringify({ sightings: [sighting] }),
}).catch(() => {}) // best-effort, fail-open
);
}
return NextResponse.next();
}Node.js (Express)
Za Express (ili bilo koji drugi Node poslužitelj) dodajte fire-and-forget middleware. On prijavljuje samo zahtjeve čiji User-Agent odgovara poznatom AI agentu ili koji nose Web Bot Auth potpis — i nikada ne blokira odgovor:
import express from "express";
const app = express();
const OBSERVE_URL = "https://api.silentshield.io/api/v1/agent/telemetry";
const API_KEY = "YOUR_API_KEY";
// Known AI-agent UA tokens. For the always-fresh list, fetch
// https://api.silentshield.io/api/v1/agent/bot-directory once a day and cache it.
const BOTS = ["GPTBot", "OAI-SearchBot", "ChatGPT-User", "ClaudeBot", "Claude-User",
"PerplexityBot", "Perplexity-User", "Bingbot", "Applebot", "Amazonbot", "CCBot",
"Bytespider", "meta-externalagent", "Google-Agent"];
// Fire-and-forget observer — never blocks the response.
app.use((req, res, next) => {
const ua = req.headers["user-agent"] || "";
const signed = !!req.headers["signature"];
if (signed || BOTS.some((b) => ua.toLowerCase().includes(b.toLowerCase()))) {
const sighting = {
ua, ip: req.ip, path: req.path, method: req.method,
...(signed && {
signature: req.headers["signature"],
signature_input: req.headers["signature-input"],
signature_agent: req.headers["signature-agent"],
authority: req.headers["host"], scheme: req.protocol,
}),
};
fetch(OBSERVE_URL, {
method: "POST",
headers: { "Content-Type": "application/json", "x-api-key": API_KEY },
body: JSON.stringify({ sightings: [sighting] }),
}).catch(() => {}); // best-effort, fail-open
}
next();
});PHP
Za čisti PHP uključite ovaj promatrač na sam vrh svojeg front controllera. Viđenje šalje tek nakon što je odgovor isporučen (fastcgi_finish_request), pa stranica nikada ne postaje sporija:
<?php
// Drop-in observer: include at the very top of your front controller.
// It runs after the response is sent (fastcgi_finish_request) so it never
// slows down the page. Reuses the same publishable key.
const SS_OBSERVE_URL = "https://api.silentshield.io/api/v1/agent/telemetry";
const SS_API_KEY = "YOUR_API_KEY";
// For the always-fresh list, cache https://api.silentshield.io/api/v1/agent/bot-directory daily.
$ss_bots = ["GPTBot","OAI-SearchBot","ChatGPT-User","ClaudeBot","Claude-User",
"PerplexityBot","Perplexity-User","Bingbot","Applebot","Amazonbot","CCBot",
"Bytespider","meta-externalagent","Google-Agent"];
register_shutdown_function(function () use ($ss_bots) {
$ua = $_SERVER["HTTP_USER_AGENT"] ?? "";
$signed = !empty($_SERVER["HTTP_SIGNATURE"]);
$isBot = $signed;
foreach ($ss_bots as $b) { if (stripos($ua, $b) !== false) { $isBot = true; break; } }
if (!$isBot) return;
if (function_exists("fastcgi_finish_request")) { fastcgi_finish_request(); }
$sighting = [
"ua" => $ua,
"ip" => $_SERVER["REMOTE_ADDR"] ?? "",
"path" => strtok($_SERVER["REQUEST_URI"] ?? "/", "?"),
"method" => $_SERVER["REQUEST_METHOD"] ?? "GET",
];
$ch = curl_init(SS_OBSERVE_URL);
curl_setopt_array($ch, [
CURLOPT_POST => true,
CURLOPT_RETURNTRANSFER => true,
CURLOPT_TIMEOUT => 3,
CURLOPT_HTTPHEADER => ["Content-Type: application/json", "x-api-key: " . SS_API_KEY],
CURLOPT_POSTFIELDS => json_encode(["sightings" => [$sighting]]),
]);
curl_exec($ch); // best-effort, fail-open
curl_close($ch);
});Go
Samostalan observer u nekoliko redaka — samo standardna biblioteka, bez SDK-a. Uočavanja se šalju fire-and-forget, pa nijedan zahtjev ne dobiva latenciju:
// Self-contained observer — stdlib only, no SDK required.
// Reports AI-agent sightings fire-and-forget: never blocks a request,
// fail-open on any error. For the always-fresh bot list, cache
// https://api.silentshield.io/api/v1/agent/bot-directory daily.
const observeURL = "https://api.silentshield.io/api/v1/agent/telemetry"
const siteKey = "YOUR_API_KEY" // publishable — or os.Getenv("SILENTSHIELD_SITE_KEY")
var bots = []string{"GPTBot", "OAI-SearchBot", "ChatGPT-User", "ClaudeBot",
"Claude-User", "PerplexityBot", "Perplexity-User", "Bingbot", "Applebot",
"Amazonbot", "CCBot", "Bytespider", "meta-externalagent", "Google-Agent"}
func observe(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
ua, signed := r.UserAgent(), r.Header.Get("Signature") != ""
hit := signed
for _, b := range bots {
if strings.Contains(strings.ToLower(ua), strings.ToLower(b)) {
hit = true
break
}
}
if hit {
ip, _, _ := net.SplitHostPort(r.RemoteAddr)
s := map[string]any{"ua": ua, "ip": ip, "path": r.URL.Path, "method": r.Method}
if signed { // Web Bot Auth (RFC 9421) — forward for verification
s["signature"] = r.Header.Get("Signature")
s["signature_input"] = r.Header.Get("Signature-Input")
s["signature_agent"] = r.Header.Get("Signature-Agent")
s["authority"] = r.Host
}
body, _ := json.Marshal(map[string]any{"sightings": []any{s}})
go func() { // best-effort, fail-open
req, _ := http.NewRequest("POST", observeURL, bytes.NewReader(body))
req.Header.Set("Content-Type", "application/json")
req.Header.Set("x-api-key", siteKey)
if resp, err := (&http.Client{Timeout: 3 * time.Second}).Do(req); err == nil {
resp.Body.Close()
}
}()
}
next.ServeHTTP(w, r)
})
}
// net/http: http.ListenAndServe(":8080", observe(mux))
// Gin: srv := &http.Server{Addr: ":8080", Handler: observe(r)}
// srv.ListenAndServe()Testirajte vezu
Ne morate čekati prvog pravog AI crawlera da biste potvrdili da vaša integracija radi. Pošaljite prazan batch — on registrira heartbeat za vaš ključ bez zapisivanja lažnih viđenja:
curl -X POST https://api.silentshield.io/api/v1/agent/telemetry \
-H "Content-Type: application/json" \
-H "x-api-key: YOUR_API_KEY" \
--data '{"sightings":[]}'Hub za agente vašeg API ključa (API ključevi → odaberite ključ → Agent) tada prelazi iz stanja „nije povezano“ u „povezano — osluškuje“. Vaše izvješće ostaje prazno dok ne dođe pravi agent: SilentShield nikada ne izmišlja podatke.
GDPR
Promatranje se temelji na legitimnom interesu (čl. 6. st. 1. toč. (f) GDPR-a). IP adrese hashiraju se na poslužiteljskoj strani sa salt vrijednošću koja se mijenja svakodnevno — sirove IP adrese nikada se ne pohranjuju. Viđenja se čuvaju 14 dana, a pohranjuju se isključivo zahtjevi poznatih AI agenata: uobičajeni promet posjetitelja nikada se ne bilježi.
FAQ
Usporava li to moju web-stranicu?
Ne. Svaka integracija radi po principu fire-and-forget: Next.js koristi event.waitUntil(), PHP šalje tek nakon isporučenog odgovora, a Node i Go šalju u pozadini. A ako SilentShield API ikada bude nedostupan, vaša stranica ostaje netaknuta — sve radi fail-open.
Što se računa kao AI agent?
Samo zahtjevi koji odgovaraju SilentShieldovu imeniku botova s poznatim AI crawlerima i agentima (GPTBot, ClaudeBot, PerplexityBot, …) ili koji nose Web Bot Auth potpis. Uvijek aktualan popis dostupan je na https://api.silentshield.io/api/v1/agent/bot-directory.
Zašto još ne vidim podatke?
AI crawleri dolaze prema vlastitom rasporedu. Na većini stranica prva se viđenja pojave unutar 24–72 sata; kod manjih stranica može potrajati dulje. U međuvremenu heartbeat u vašem hubu za agente potvrđuje da sama integracija radi.