Politika duyarlı text-to-SQL ajanının postgresql satır düzeyi güvenlik (RLS) ile uygulamalı değerlendirilmesi
Applied evaluation of the policy-sensitive text-to-SQL agent with postgresql row level security (RLS)
- Tez No: 1018231
- Danışmanlar: PROF. DR. İSMAİL HAKKI CEDİMOĞLU
- Tez Türü: Yüksek Lisans
- Konular: Bilgisayar Mühendisliği Bilimleri-Bilgisayar ve Kontrol, Computer Engineering and Computer Science and Control
- Anahtar Kelimeler: Belirtilmemiş.
- Yıl: 2026
- Dil: Türkçe
- Üniversite: Sakarya Üniversitesi
- Enstitü: Fen Bilimleri Enstitüsü
- Ana Bilim Dalı: Bilişim Sistemleri Mühendisliği Ana Bilim Dalı
- Bilim Dalı: Belirtilmemiş.
- Sayfa Sayısı: Belirtilmemiş.
Özet
Günümüz dijital dönüşüm sürecinde, büyük dil modelleri (BDM) doğal dil sorgularını SQL ifadelerine dönüştürerek veri erişimini demokratikleştirmekte, ancak kurumsal ortamlarda erişim kontrolü ve veri güvenliği açısından önemli riskler doğurmaktadır. Bu tez, BDM tabanlı Text-to-SQL ajanlarının politika duyarlı üretiminde, erişim kısıtlarının yalnızca istem (prompt) düzeyinde ifade edildiği“yumuşak”güvenlik yaklaşımının (Prompt_Only) yetersizliğini ve PostgreSQL Satır Düzeyi Güvenliği (RLS) ile entegre edilmiş“sert”güvenlik yaklaşımının (RLS_Agent) üstünlüğünü karşılaştırmalı olarak incelemektedir. Çalışma, BDM'lerin probabilistik doğası nedeniyle talimat ihlali, istem enjeksiyonu ve veri sızıntısı gibi sorunları ele almakta; bu risklerin özellikle çok kiracılı veritabanlarında, farklı kullanıcı rollerine (yönetici, bölge direktörü, uyum sorumlusu, proje lideri, analist) göre erişim haklarının yönetildiği senaryolarda kritikleştiğini vurgulamaktadır. Araştırmanın amacı, deterministik zorlamanın politika uyumunu (PCR), veri sızıntısını (DLV) ve yürütüm doğruluğunu (EA) nasıl etkilediğini deneysel olarak kanıtlamaktır. Bu doğrultuda üç araştırma sorusu tanımlanmıştır: (1) RLS ile zorlanan modelde güvenlik metrikleri ne düzeyde kontrol edilebilmektedir? (2) Prompt_Only yaklaşımı hangi riskleri üretmektedir? (3) Güvenlik katmanının veritabanına taşınması performans açısından ne tür etkilere sahiptir? Buna karşılık üç hipotez test edilmiştir: (H1) RLS_Agent'in DLV'yi sıfıra yaklaştıracağı ve PCR'yi anlamlı biçimde yükselteceği, (H2) Prompt_Only'un yüksek riskli senaryolarda anlamlı DLV ve düşük PCR üreteceği, (H3) Gecikme farkının model davranışından etkileneceği. Yöntemsel olarak, çalışma PostgreSQL üzerinde kurumsal bir şema (çalışan kayıtları, projeler, atamalar) ve karma erişim politikaları (öznitelik tabanlı - ABAC, çok seviyeli güvenlik - MLS, ilişki tabanlı - ReBAC) ile sınırlıdır. RLS_Agent, BDM (GPT-5-mini) tarafından üretilen SQL sorgularını doğrulayıp rol bazlı bağlamda yürüterek RLS ile satır görünürlüğünü sınırlandırmakta; Prompt_Only ise kısıtları yalnızca metinsel talimatlarla yönetmektedir. Deneyler, ALLOW (izinli) ve DENY (yasaklı) senaryolarını kapsayan 13 test case üzerinden N=30 tekrar ile gerçekleştirilmiş; toplam 780 run kaydedilmiştir. Metrikler, politika uyumu (PCR: uyumlu run oranı), veri sızıntısı (DLV: yetkisiz ifşa büyüklüğü), yürütüm doğruluğu (EA: ölçülebilir çıktı oranı) ve gecikme (LLM/DB/E2E) bileşenleri üzerinden hesaplanmıştır. Bulgular, RLS_Agent'in genel PCR'yi %99.74'e yükselttiğini, DLV'yi sıfıra yaklaştırdığını ve EA'yi %92.05 düzeyinde tuttuğunu göstermektedir. Buna karşılık Prompt_Only, kritik senaryolarda PCR'yi %73.59'a düşürmüş, DLV'yi ortalama 482 birim düzeyine çıkarmış ve EA'yi %90.26'da sınırlamıştır. Özellikle analist ve uyum rollerinde Prompt_Only sistematik sızıntı üretmiş; RLS_Agent ise deterministik filtreleme ile bu riski ortadan kaldırmıştır. Gecikme analizinde, E2E sürenin baskın olarak LLM kaynaklı olduğu (RLS_Agent: 4.38 s, Prompt_Only: 6.96 s) ve Prompt_Only'un bazı senaryolarda daha yüksek oynaklık sergilediği gözlenmiştir. Bu sonuçlar, H1 ve H2 hipotezlerini güçlü biçimde doğrulamakta; H3'ü ise gecikme farkının model istikrarsızlığından etkilendiği yönünde desteklemektedir. Sonuç olarak, tez BDM tabanlı Text-to-SQL sistemlerinin kurumsal benimsenmesinde güvenlik katmanının modelden bağımsızlaştırılmasının gerekliliğini vurgulamakta; RLS entegrasyonunun hem güvenlik hem performans açısından pratik bir çözüm sunduğunu ortaya koymaktadır. Katkılar, güvenilir AI kavramına nicel metrikler (PCR/DLV) ile katkı sağlamakta; sınırlılıklar ise tek model kullanımı ve şema ölçeğiyle ilişkilendirilmektedir. Gelecek çalışmalar, farklı veritabanı sistemleri ve dinamik politikalarla genişletilebilir.
Özet (Çeviri)
In the contemporary era of digital transformation, large language models (LLMs) have revolutionized natural language processing, particularly in critical applications such as database interfaces and data access. LLMs facilitate the automation of Text-to-SQL tasks by converting users' natural language queries into SQL statements, thereby making data analysis more accessible to non-experts. This evolution has shifted from traditional semantic parsing methods to context-aware, learning-based approaches, enabling effective query generation even on undefined schemas through zero-shot capabilities (Dou et al., 2024). However, despite these advancements, the probabilistic nature of LLM-based systems poses significant challenges in enterprise environments concerning security and access control. In multi-tenant databases where different user roles (e.g., administrator, analyst, compliance officer) have varying access privileges, the model's tendency to deviate from instructions can lead to unauthorized data exposure. This security vulnerability stems from the inherent weakness of“soft”security approaches, where access restrictions are expressed solely through textual instructions in prompts. LLMs, due to their internal architectures, exhibit non-deterministic behaviors, such as violating or misinterpreting directives, which can facilitate access to sensitive data (Zou et al., 2023). This issue is exacerbated by prompt injection attacks, where user inputs manipulate the model to override prior constraints, resulting in data leaks (Perez & Ribeiro, 2023; Liu et al., 2024). In corporate contexts, these risks materialize concretely; for instance, an analyst role limited to viewing their department's employee records might inadvertently access company-wide data if the model generates overly broad queries. Such scenarios not only compromise data privacy but also threaten legal compliance (e.g., GDPR-like regulations) and organizational trust (Sun et al., 2024). As highlighted in the literature, the accuracy-focused development of LLMs has left the security layer dependent on model behavior, potentially leading to catastrophic outcomes in practical Text-to-SQL agent deployments (Greshake et al., 2023). Furthermore, this problem extends beyond a mere technical flaw, questioning the foundational principles of enterprise data management. Traditional database systems enforce access control through deterministic mechanisms (e.g., role-based access), whereas LLM integration blurs these boundaries. The model's“reasoning”capability may generalize or ignore instructions; for example, a“only your region”constraint could be misinterpreted as“all regions.”This ambiguity elevates data leakage risks, especially in environments handling sensitive information like salary details or project secrets. The thesis contrasts this with the Prompt_Only approach, emphasizing the necessity of hard security mechanisms like RLS. Ultimately, while the potential of LLM-based Text-to-SQL is high, its security gaps remain the primary barrier to enterprise-scale adoption (Sandhu, 2022). The primary objective of this study is to comparatively examine two security approaches in policy-aware Text-to-SQL generation, demonstrating experimentally the superiority of deterministic enforcement of access constraints. The proposed RLS_Agent method integrates LLM-generated SQL queries with PostgreSQL Row-Level Security (RLS), rendering the security layer independent of the model's probabilistic behavior. This approach executes queries under simulated user roles, deterministically limiting row visibility to prevent data leaks (Derakhshan et al., 2023; PostgreSQL Global Development Group, 2024). The comparison is made against the Prompt_Only baseline, where constraints are defined solely at the prompt level, introducing risks tied to model non-compliance. The evaluation focuses on policy compliance rate (PCR), data leakage value (DLV), and execution accuracy (EA) metrics. The significance of the research arises from the growing security needs amid the rapid proliferation of LLM-based systems in enterprises. Current literature predominantly emphasizes accuracy and error correction in Text-to-SQL (Gao et al., 2023; Li et al., 2024); however, this work foregrounds the security dimension, quantifying policy adherence through novel metrics. For example, the DLV metric quantifies breach magnitude, highlighting the practical risks of soft approaches and proposing a new standard for enterprise data protection. Additionally, the study contributes to trustworthy AI by relocating security boundaries to the database layer, mitigating catastrophic risks from model uncertainty and promoting reliable LLM usage (Bertino & Kundu, 2023). Practically, these findings guide system designers in sensitive sectors like healthcare or finance, where RLS integration can reduce data leaks to near-zero. Moreover, the study's importance extends to societal and ethical dimensions. Data breaches erode individual privacy and trust; thus, deterministic security mechanisms support responsible AI deployment. The thesis concretizes this contribution through PCR and DLV metrics, filling a gap in literature: unlike accuracy-focused studies, it balances security and performance (Gu et al., 2024). In conclusion, this research shapes the future of LLM-based agents, offering both theoretical and applied value. This work aims to evaluate policy-aware Text-to-SQL agents in terms of security and accuracy, addressing the following research questions: RQ1: Under an RLS-enforced role/policy model, to what extent can the Text-to-SQL agent control policy compliance (PCR) and data leakage (DLV)? This question assesses the efficacy of deterministic enforcement, highlighting advantages of a model-independent security layer. RQ2: In the Prompt_Only approach where role constraints are defined solely at the prompt level, what risks does it produce in terms of policy violations and data leakage in similar scenarios? This question exposes the structural weaknesses of soft security, quantifying practical risks. RQ3: What costs or gains does shifting security enforcement to the database layer impose on execution accuracy (EA) and latency? This question evaluates the performance impact of security enhancements, providing a holistic analysis. Corresponding hypotheses tested in the experimental design are defined as follows: H1 (Deterministic Enforcement Hypothesis): Under RLS_Agent, data leakage value (DLV) will approach zero across all test scenarios, and policy compliance rate (PCR) will be significantly higher than Prompt_Only. This assumes database-layer enforcement neutralizes model non-compliance. H2 (Prompt Reliability Hypothesis): Prompt_Only, relying on textual instructions for role constraints, will produce significant DLV > 0 and reduced PCR, especially in high-risk scenarios (e.g., broad listing/count requests, sensitivity-labeled rows). This emphasizes the probabilistic nature of prompt-based constraints. H3 (Performance Hypothesis): Since LLM latency dominates end-to-end (E2E) delay, database latencies will be similar; total latency differences will primarily stem from model output stability and validation/retry behaviors. This expects security layering to impose limited performance costs. These questions and hypotheses define the experimental focus, enhancing the interpretability of results. The scope is limited to a corporate sample schema on PostgreSQL, including tables like employee records, projects, and assignments. Access policies combine attribute-based (ABAC), multi-level security (MLS), and relation-based (ReBAC-like) models. Roles represent varying access levels: administrator (broad authority), region director (ABAC-focused), compliance user (MLS-focused), project lead (ReBAC-focused), and analyst (restricted access). The model infrastructure uses OpenAI GPT-5-mini, selected for low latency and high accuracy in comparative analyses (Li et al., 2024). Experiments cover ALLOW and DENY scenarios, measuring security metrics (PCR, DLV) and performance indicators (EA, latency). Limitations are explicitly stated to maintain objectivity. First, using a single model (GPT-5-mini) may limit generalizability to other LLMs (e.g., larger models), necessitating future exploration of model diversity. Second, while the schema and policy set simulate an enterprise environment, real-world databases' complexity (e.g., millions of rows) is not represented, posing scalability constraints. Third, the Prompt_Only baseline's assumption of“over-privileged”execution context models practical misconfigurations but overlooks additional layers (e.g., SQL injection filters). Additionally, experiments conducted in a specific timeframe may be affected by LLM updates altering behaviors. These limitations condition the findings but provide a foundation for future extensions (e.g., diverse databases or models) and uphold academic integrity. The literature review covers the evolution of Text-to-SQL and agent-based frameworks, security risks and prompt injection vulnerabilities, data-layer enforcement and RLS, and a synthesis identifying research gaps. Text-to-SQL has evolved from simple semantic parsing to autonomous LLM-based agent frameworks, democratizing data access. Traditional methods relied on predefined schemas, but LLMs offer flexible, context-sensitive handling (Dou et al., 2024). Li et al. (2024) evaluate LLMs as database interfaces, noting superior generalization but persistent security and consistency issues, validating the inadequacy of prompt-level role constraints in this thesis. Agent frameworks automate error correction via execution feedback (Gao et al., 2023), aligning with RLS_Agent's execution-guided refinement while incorporating deterministic security. Security risks in LLM-based Text-to-SQL stem from soft approaches' fragility. Prompt injection enables overriding instructions (Perez & Ribeiro, 2023; Liu et al., 2024), leading to unauthorized queries. Model alignment vulnerabilities allow adversarial triggers to elicit undesired behaviors (Zou et al., 2023; Greshake et al., 2023), supporting Prompt_Only's risks in policy breaches and leaks. Data-layer enforcement addresses these by making access deterministic. Sandhu (2022) discusses access control in big data/AI eras, advocating dynamic policies. Derakhshan et al. (2023) detail RLS for multi-tenant apps, aligning with this thesis's policy models. PostgreSQL documentation (2024) outlines RLS implementation, enabling row visibility restriction independent of queries. The synthesis reveals literature's accuracy focus (Dou et al., 2024; Gao et al., 2023), underaddressing security metrics like PCR/DLV. This thesis fills the gap by comparing RLS_Agent and Prompt_Only, positioning PCR/DLV as trustworthy AI metrics (Sun et al., 2024). The methodology compares policy/role-aware Text-to-SQL generation under two approaches, addressing unauthorized data exposure in enterprises with varying role privileges. RLS_Agent (hard enforcement/DB-layer security) validates and executes LLM-generated SQL under simulated roles with RLS limiting visibility deterministically. Prompt_Only (prompt-based/soft security) defines constraints textually, risking leaks from model deviations or over-privileged contexts. Research questions and hypotheses are reiterated as above. Scope limits to PostgreSQL schema with hybrid policies; OpenAI GPT-5-mini selected for efficiency (Fagerberg et al., 2025). System model includes data layer (PostgreSQL+RLS), agent layer (Text-to-SQL generation), and execution layer (role-based connection). Threat model for Prompt_Only covers instruction violation, injection, over-privileged execution; for RLS_Agent, unauthorized access attempts are neutralized by RLS. Schema design principles feature employee records (ABAC/MLS), projects, and assignments (ReBAC). Roles differentiate policy types and risk surfaces. RLS policies enforce deterministic visibility, enabling systematic scenario generation. RLS_Agent's principle ensures query safety (SELECT subset) and access security (role-based execution with RLS). Components include input preparation, prompt construction, LLM call, SQL validation/sanitization, role-based execution, output normalization, and logging. Prompt integrates schema, role, output mode (scalar/rows). Validation rules ensure single-statement SELECT, no side effects. The comparative design isolates enforcement differences over identical tasks. Scenarios cover role/policy diversity, ALLOW/DENY, scalar/rows outputs, testing ABAC/MLS/ReBAC. Results from 780 runs (13 scenarios × 2 methods × 30 repeats) show RLS_Agent's superior PCR (99.74%), zero DLV, and EA (92.05%). Prompt_Only yields PCR (73.59%), high DLV (482 average), EA (90.26%). Role-based summaries highlight Prompt_Only failures in compliance/analyst roles. Detailed scenario results confirm deterministic RLS mitigation of leaks. Latency dominated by LLM, with Prompt_Only showing higher variance in critical cases. Token costs approximate higher for Prompt_Only. In conclusion, the thesis underscores the need for model-independent security in LLM-based Text-to-SQL, with RLS offering a practical solution. Contributions advance trustworthy AI; limitations include model singularity. Future work could explore dynamic policies and diverse systems.
Benzer Tezler
- Fransa (Sfio-ps) örneğinde Batı Avrupa Sosyal Demokrat Partililerin dış politika eğilimleri
Başlık çevirisi yok
ŞULE GÖNEN (DİNMEZ)
- İmara ilişkin bağışlamaların gelişimi ve değerlendirilmesi
Evoution et evaluation des amnisties concernant le constructions et le development du territoire
AHMET LÜTFİ UZEL
Doktora
Türkçe
1986
Şehircilik ve Bölge PlanlamaGazi ÜniversitesiŞehir ve Bölge Planlama Ana Bilim Dalı
DOÇ. DR. İBRAHİM BOYNUKALIN
- Çatışmacı toplum paradigması: Teorik ve meta-teorik düzeylerin ilişkisi
Conflict paradigm of society
ADNAN TÜREGİN