Understanding Solaris Kernel Stack Overflows (Doc ID 1011675.1)

Goal

How to use this document

Kernel stack overflows are a complex topic, so this document includes a fair amount of discussion. You should read and understand it all before making any modifications to your system, careless tuning could render your system unusable. For the impatient, section 4 contains all the suggestions for avoiding stack overflows - but be sure you understand the side-effects of any parameters you change. If in doubt, read the earlier sections!

1.1 Introduction

1.2 Terminology

2.1 The Difficulty of Dynamically Growing Kernel Stacks

2.2 Stack Usage Rules For Kernel Developers

3.1 Kernel Stacks In Solaris

3.1.1 Stack Sizes for Different Kinds of Threads

3.2 What Does a Kernel Stack Overflow Look Like

3.3 Past Causes of Stack Overflow in Solaris

3.3.1 M64 Graphics Driver ati

3.3.2 setcontext()

3.3.3 Pagefault Handling

3.3.4 Recursion in prrealvp

3.4 Mapping the Red Zone

4.1 Preventing Kernel Stack Overflows in Solaris

4.1.1 Kernel Parameters Available for Tuning in Solaris 8 and 9

4.1.2 Kernel Parameters Available for Tuning in Solaris 10

4.1.3 Consequences of Tuning Stack Sizes

4.1.4 Recommendations

4.1.5 Confirming Changes to Stack Size Parameters

Solution