Understanding Solaris Kernel Stack Overflows

(Doc ID 1011675.1)

Last updated on AUGUST 01, 2016

Applies to:

Solaris Operating System - Version 8.0 and later
Information in this document applies to any platform.
***Checked for relevance on 03-Jul-2014***

Goal

How to use this document

Kernel stack overflows are a complex topic, so this document includes a fair amount of discussion. You should read and understand it all before making any modifications to your system, careless tuning could render your system unusable. For the impatient, section 4 contains all the suggestions for avoiding stack overflows - but be sure you understand the side-effects of any parameters you change. If in doubt, read the earlier sections!

1.1 Introduction

1.2 Terminology

2.1 The Difficulty of Dynamically Growing Kernel Stacks

2.2 Stack Usage Rules For Kernel Developers

3.1 Kernel Stacks In Solaris

3.1.1 Stack Sizes for Different Kinds of Threads

3.2 What Does a Kernel Stack Overflow Look Like

3.3 Past Causes of Stack Overflow in Solaris

3.3.1 M64 Graphics Driver ati

3.3.2 setcontext()

3.3.3 Pagefault Handling

3.3.4 Recursion in prrealvp

3.4 Mapping the Red Zone

4.1 Preventing Kernel Stack Overflows in Solaris

4.1.1 Kernel Parameters Available for Tuning in Solaris 8 and 9

4.1.2 Kernel Parameters Available for Tuning in Solaris 10

4.1.3 Consequences of Tuning Stack Sizes

4.1.4 Recommendations

4.1.5 Confirming Changes to Stack Size Parameters

Solution

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms