各位學(xué)員好，前天有個《python信用評分卡建模（附代碼）》課程學(xué)員向我提問。她在銀行工作，銀行對金融風(fēng)控模型驗證和壓力測試非常重視。目前不清楚如果對風(fēng)控模型壓力測試。

這個問題很專業(yè)，很少有學(xué)生提到這問題。目前國內(nèi)對風(fēng)控模型的驗證并不重視，只有銀行和少數(shù)持牌照消費金融公司有模型驗證團隊。有的模型驗證團隊并不正規(guī)，難以保持工作獨立性。

國內(nèi)銀行參考了美國美聯(lián)儲SR 11-7文件Guidance on Model Risk Management，翻譯為中文為模型風(fēng)險管理指南。

美聯(lián)儲2011年4月發(fā)布模型風(fēng)險管理的相關(guān)法規(guī)——SR 11-7號文《模型風(fēng)險管理指南》，該文明確了模型風(fēng)險管理框架，包括模型風(fēng)險管理定義、模型實施、模型驗證、以及模型風(fēng)險管理的政策制度等內(nèi)容。SR 11-7號文重點在模型驗證環(huán)節(jié)。

回到正題，模型壓力和風(fēng)險測試很難，當(dāng)客群變化后，模型穩(wěn)定性下降，即PSI指數(shù)上升，模型是需要重新迭代的。當(dāng)發(fā)生疫情后，如果失業(yè)率快速上升，客戶違約風(fēng)險也會快速上升。這不是子模型能解決的問題，需要建立宏觀經(jīng)濟模型來提前預(yù)測失業(yè)率等指標(biāo)，然后收緊策略，縮小放款范圍。

至于高風(fēng)險客戶，模型還是有辦法的。我推薦邏輯回歸評分卡模型建模，評分卡的woe分箱中，可以把缺失值或異常值(高風(fēng)險值)單獨分箱。因此評分卡模型對異常值和缺失值有很高容忍度。感興趣朋友可以收藏和關(guān)注我的課程《python信用評分卡建模（附代碼）》。該課程還有模型驗證的常用方法，包括模型區(qū)分能力，排序能力，穩(wěn)定性驗證方法。

不幸的是，國內(nèi)諸多金融公司在建模中存在結(jié)構(gòu)性問題。例如一個申請單進(jìn)入審批，策略會先評估該客戶信用風(fēng)險，如果沒問題才進(jìn)入模型判斷。模型幾乎是墊底的。這意味著諸多變量的異常值已經(jīng)被提前過濾了。這種模型是非常脆弱的。因此我還是建議大家把異常值，高風(fēng)險值加入模型訓(xùn)練，提高模型抗壓能力。畢竟評分卡模型可以對異常值，高風(fēng)險值單獨分箱處理。

大家不要把模型想的很復(fù)雜，你可以把模型想成一個人。這個人在平時訓(xùn)練時經(jīng)歷過大風(fēng)大浪，才有抗壓風(fēng)險。如果這個人經(jīng)歷太順利，可能遇到一個挫折就不知所措。

對于壓力測試，我還想談?wù)劥蠹覍Ξ惓Ｖ嫡J(rèn)真的誤區(qū)。

很多建模人員遇到異常值時，直接刪除異常值，然后建模。對于壓力測試，這是不妥的。很多高風(fēng)險值就是異常值。當(dāng)欺詐客戶行騙時，就會觸發(fā)某些變量產(chǎn)生異常值。如果我們輕易刪除異常值，就不會捕捉到這些欺詐客戶，而且模型區(qū)分能力可能降低。《python信用評分卡建模（附代碼）》的give me some credit案例章節(jié)就有去掉異常值后，模型區(qū)分能力下降實驗測試。

模型風(fēng)險管理還有個難點就是人禍。經(jīng)營者在面對巨額利潤時，會刻意忽略風(fēng)險。這也是造成風(fēng)控部門和業(yè)務(wù)部門長期對立原因。業(yè)務(wù)部門想賺更多錢，讓風(fēng)控部門放松策略。風(fēng)控部門想降低風(fēng)險，收緊策略。而且業(yè)務(wù)部門和風(fēng)控部門考核KPI指標(biāo)是相反的。

之前發(fā)布過文章《捷信金融（Home Credit N.V.）興衰如夢，留下寶貴機器學(xué)習(xí)建模數(shù)據(jù)》。捷信作為國內(nèi)最早消費金融公司，擁有大批風(fēng)控專家，其風(fēng)控策略和風(fēng)控模型是非常強悍的。隨著捷信線下迅速擴張，造成風(fēng)控跟不上和故意對風(fēng)險視而不見，最后捷信老大哥跌下神壇。

電影《大空頭》給觀眾解釋了美國華爾街金融危機爆發(fā)前，諸多數(shù)據(jù)分析師已經(jīng)察覺風(fēng)險。但美國政客，金融從業(yè)人員甚至風(fēng)險監(jiān)管人員為了高額利潤對其視而不見，造成后來全球巨大經(jīng)融危機，甚至到今天問題都沒解決。大家可以去看看這部經(jīng)典電影。

最后我把SR 11-7：模型風(fēng)險管理指南（Guidance on Model Risk Management）中英文放入文章，供學(xué)員參考。

致各聯(lián)邦儲備銀行負(fù)責(zé)監(jiān)督的官員和適當(dāng)?shù)谋O(jiān)督檢查人員

主題：

模型風(fēng)險管理指南

美聯(lián)儲和貨幣監(jiān)理署 (OCC)正在發(fā)布隨附的模型風(fēng)險管理監(jiān)管指南，供銀行組織和監(jiān)管機構(gòu)在評估組織對模型風(fēng)險的管理時使用。本指南應(yīng)酌情適用于受美聯(lián)儲監(jiān)管的所有銀行組織，同時考慮每個組織的規(guī)模、性質(zhì)和復(fù)雜性，以及模型使用的范圍和復(fù)雜程度（定義和討論如下）。

模型風(fēng)險管理

銀行組織應(yīng)注意基于不正確或濫用模型的決策可能產(chǎn)生的不利后果（包括財務(wù)損失），并應(yīng)通過積極的模型風(fēng)險管理來解決這些后果。此SR 信的附件更詳細(xì)地描述了有效模型風(fēng)險管理框架的關(guān)鍵方面，包括穩(wěn)健的模型開發(fā)、實施和使用；有效的驗證；健全的治理、政策和控制。

美聯(lián)儲和 OCC 之前發(fā)布的出版物已經(jīng)討論了模型的使用，特別關(guān)注模型驗證。1基于過去幾年的監(jiān)管和行業(yè)經(jīng)驗，本文件擴展了現(xiàn)有指南——最重要的是擴大范圍以包括模型風(fēng)險管理的其他關(guān)鍵方面。

就本文檔而言，術(shù)語模型是指應(yīng)用統(tǒng)計、經(jīng)濟、金融或數(shù)學(xué)理論、技術(shù)和假設(shè)將輸入數(shù)據(jù)處理為定量估計的定量方法、系統(tǒng)或方法。符合此定義的模型可用于分析業(yè)務(wù)戰(zhàn)略、為業(yè)務(wù)決策提供信息、識別和衡量風(fēng)險、評估風(fēng)險敞口、工具或頭寸、進(jìn)行壓力測試、評估資本充足性、管理客戶資產(chǎn)、衡量內(nèi)部限制的合規(guī)性、維護(hù)正式銀行的控制設(shè)備，或滿足財務(wù)或監(jiān)管報告要求并發(fā)布公開披露信息。模型的定義還包括定量方法，其輸入部分或全部是定性的或基于專家判斷，前提是輸出是定量的。2

模型的使用總是會帶來模型風(fēng)險，即基于不正確或濫用模型輸出和報告的決策可能產(chǎn)生不利后果。模型風(fēng)險可能導(dǎo)致財務(wù)損失、糟糕的業(yè)務(wù)和戰(zhàn)略決策，或損害銀行組織的聲譽。模型風(fēng)險的發(fā)生主要有兩個原因：??(1)模型可能存在根本性錯誤并在與其設(shè)計目標(biāo)和預(yù)期業(yè)務(wù)用途相悖時產(chǎn)生不準(zhǔn)確的輸出；(2) 一個模型可能被錯誤或不恰當(dāng)?shù)厥褂?，或者可能對其局限性和假設(shè)存在誤解。模型風(fēng)險隨著模型復(fù)雜性的增加、輸入和假設(shè)的更高不確定性、更廣泛的使用范圍和更大的潛在影響而增加。銀行組織應(yīng)該從單個模型和整體模型中管理模型風(fēng)險。

貫穿整個指南的指導(dǎo)原則是管理模型風(fēng)險涉及模型的“有效挑戰(zhàn)”：由客觀、知情的各方進(jìn)行批判性分析，可以識別模型限制并產(chǎn)生適當(dāng)?shù)淖兓?/span>有效的挑戰(zhàn)取決于激勵、能力和影響力的結(jié)合。

與其他風(fēng)險一樣，重要性是模型風(fēng)險管理中的一個重要考慮因素。如果在某些銀行，模型的使用不那么普遍并且對其財務(wù)狀況的影響較小，那么這些銀行可能不需要如此復(fù)雜的模型風(fēng)險管理方法來滿足監(jiān)管預(yù)期。但是，如果模型和模型輸出對業(yè)務(wù)決策（包括與風(fēng)險管理和資本和流動性規(guī)劃相關(guān)的決策）產(chǎn)生重大影響，并且模型失效會對銀行的財務(wù)狀況產(chǎn)生特別有害的影響，銀行的模型風(fēng)險管理框架應(yīng)該更加廣泛和嚴(yán)謹(jǐn)。

模型開發(fā)、實施和使用

模型開發(fā)很大程度上依賴于開發(fā)人員的經(jīng)驗和判斷，模型風(fēng)險管理應(yīng)包括與模型用戶的情況和目標(biāo)以及銀行組織政策相一致的規(guī)范的模型開發(fā)和實施過程。完善的開發(fā)過程包括：明確的目的聲明，以確保模型的開發(fā)符合其預(yù)期用途；模型背后的合理設(shè)計、理論和邏輯；穩(wěn)健的模型方法和處理組件；嚴(yán)格評估數(shù)據(jù)質(zhì)量和相關(guān)性；和適當(dāng)?shù)奈募?/span>模型開發(fā)的一個組成部分是測試，其中評估模型的各個組件及其整體功能，以顯示模型按預(yù)期執(zhí)行；證明它是準(zhǔn)確、穩(wěn)健和穩(wěn)定的；并評估其局限性和假設(shè)。重要的是，組織應(yīng)確保其模型更具判斷性和定性的方面的開發(fā)也是合理的。

所有模型都有一定程度的不確定性和不準(zhǔn)確性，因為它們根據(jù)定義是對現(xiàn)實的不完美表示。有效模型開發(fā)、實施和使用的一個重要結(jié)果是銀行組織對這種不確定性的理解和解釋。對模型不確定性的考慮可以包括對模型輸出應(yīng)用有充分支持的、判斷性的、“保守的”調(diào)整，對模型輸出的重視程度較低，或確保僅在有其他模型或方法補充時才使用模型。3

模型驗證

模型驗證是一組過程和活動，旨在驗證模型是否按預(yù)期執(zhí)行，符合其設(shè)計目標(biāo)和業(yè)務(wù)用途。有效的驗證有助于確保模型是合理的，識別潛在的限制和假設(shè)并評估它們可能產(chǎn)生的影響。所有模型組件——輸入、處理、輸出和報告——都應(yīng)經(jīng)過驗證；這同樣適用于內(nèi)部開發(fā)的模型以及從供應(yīng)商或顧問處購買或開發(fā)的模型。

驗證涉及在一定程度上獨立于模型開發(fā)和使用。通常，驗證由不負(fù)責(zé)模型開發(fā)或使用的人員完成，并且與模型是否被確定為有效無關(guān)。實際上，一些驗證工作可能由模型開發(fā)人員和用戶最有效地完成；然而，此類驗證工作必須接受獨立方的嚴(yán)格審查，該獨立方應(yīng)開展額外活動以確保適當(dāng)驗證。總體而言，驗證過程的質(zhì)量由客觀、知識淵博的各方的嚴(yán)格審查以及為解決這些各方確定的問題所采取的行動來表明。

在模型投入使用后，驗證活動應(yīng)持續(xù)進(jìn)行，以跟蹤已知模型限制并識別任何新限制。在經(jīng)濟和金融狀況良好的時期，驗證是一項重要的檢查，此時風(fēng)險和潛在損失的估計可能變得過于樂觀，手頭的數(shù)據(jù)可能無法完全反映壓力更大的狀況。銀行組織應(yīng)該對每個模型進(jìn)行定期審查——至少每年一次，但如果有必要，可以更頻繁地審查，以確定它是否按預(yù)期工作以及現(xiàn)有的驗證活動是否足夠。全面驗證的關(guān)鍵要素包括：

• 概念合理性評估。該要素涉及評估模型設(shè)計和構(gòu)建的質(zhì)量，以及審查支持模型所用方法和所選變量的文檔和經(jīng)驗證據(jù)。驗證的這一步驟應(yīng)確保在模型設(shè)計和構(gòu)建中做出的判斷是知情的、經(jīng)過仔細(xì)考慮的，并與已發(fā)表的研究和良好的行業(yè)實踐相一致。

• 持續(xù)監(jiān)測。驗證中的這一步是為了確認(rèn)模型得到了適當(dāng)?shù)膶嵤?，并且正在按預(yù)期使用和執(zhí)行。必須評估產(chǎn)品、風(fēng)險敞口、活動、客戶或市場條件的變化是否需要調(diào)整、重新開發(fā)或更換模型，并驗證模型超出其原始范圍的任何擴展是否有效。在此步驟中可以使用基準(zhǔn)測試將給定模型的輸入和輸出與備選方案的估計值進(jìn)行比較。

• 結(jié)果分析。此步驟涉及將模型輸出與相應(yīng)的實際結(jié)果進(jìn)行比較。回溯測試是結(jié)果分析的一種形式，它涉及在模型開發(fā)中未使用的樣本時間段內(nèi)以與模型的預(yù)測范圍或性能窗口相匹配的頻率將實際結(jié)果與模型預(yù)測進(jìn)行比較。

驗證過程的三個核心要素的結(jié)果可能會揭示模型開發(fā)中的重大錯誤或不準(zhǔn)確之處或始終超出銀行組織預(yù)定可接受閾值的結(jié)果。在這種情況下，需要進(jìn)行模型調(diào)整、重新校準(zhǔn)或重新開發(fā)。有時，由于缺乏數(shù)據(jù)或價格可觀察性等各種原因，銀行組織可能無法使用關(guān)鍵模型驗證工具。在這種情況下，在考慮模型使用的適當(dāng)性時，更應(yīng)注意模型的局限性，并在使用模型進(jìn)行決策時充分告知高級管理層這些局限性。一般來說，高級管理層應(yīng)確保根據(jù)已識別的模型限制采取適當(dāng)?shù)木徑獯胧?/span>

治理、政策和控制

對模型風(fēng)險管理框架開發(fā)和維護(hù)強有力的治理對其有效性至關(guān)重要。強有力的治理通過定義相關(guān)風(fēng)險管理活動的政策、實施這些政策的程序、資源分配以及測試政策和程序是否按規(guī)定執(zhí)行的機制，為風(fēng)險管理職能提供明確的支持和結(jié)構(gòu)。強有力的治理還包括足夠詳細(xì)的模型開發(fā)和驗證文檔，以使不熟悉模型的各方能夠了解模型的運行方式及其局限性和關(guān)鍵假設(shè)。

董事會和高級管理層在建立全組織范圍的模型風(fēng)險管理方法時，在最高級別提供模型風(fēng)險治理。董事會成員應(yīng)確保模型風(fēng)險水平在他們的容忍范圍內(nèi)。銀行組織的內(nèi)部審計職能應(yīng)評估模型風(fēng)險管理框架的整體有效性，包括框架處理單個模型和總體模型風(fēng)險的能力。每當(dāng)銀行組織使用外部資源進(jìn)行模型風(fēng)險管理時，組織應(yīng)在明確書面和商定的工作范圍中指定要開展的活動，并且這些活動應(yīng)按照本指南進(jìn)行。此外，組織應(yīng)維護(hù)已實施的模型清單以供使用，

所有銀行組織都應(yīng)確保其內(nèi)部政策和程序與本指南中包含的風(fēng)險管理原則和監(jiān)管預(yù)期相一致。

聯(lián)系人

有關(guān)本指南的問題，請致電(202) 452-2904聯(lián)系高級金融監(jiān)管分析師David Palmer ；Dwight Smith，資本與監(jiān)管政策高級監(jiān)管金融分析師，電話：(202) 452-2773；或助理主任Anna Lee Hewko，電話(202) 530-6260。此外，可以通過委員會的公共網(wǎng)站發(fā)送問題。4

Patrick M. Parkinson?銀行?監(jiān)管
司司長簽字

附件：

模型風(fēng)險管理指南(PDF)

交叉參考：

• SR 09-1，“市場風(fēng)險規(guī)則在銀行控股公司和國有銀行中的應(yīng)用”

筆記：

1. 例如，OCC 在OCC 2000-16?（2000 年 5 月 30 日）、其他公告和主計長手冊的某些主題小冊子中提供了關(guān)于模型風(fēng)險的指導(dǎo)，重點是模型驗證。美聯(lián)儲發(fā)布了SR 信 09-01，?“市場風(fēng)險規(guī)則在銀行控股公司和國有銀行中的應(yīng)用”，其中強調(diào)了與模型風(fēng)險管理相關(guān)的各種概念，包括驗證和審查標(biāo)準(zhǔn)、模型驗證文檔和回-測試。美聯(lián)儲的交易和資本市場活動手冊也討論了驗證和模型風(fēng)險管理。此外，高級方法基于風(fēng)險的資本規(guī)則（12 CFR 3，?附錄 C；?12 CFR 208，?附錄 F；和12 CFR 225，?附錄 G)包含針對主題銀行組織的明確驗證要求。??

2. 雖然不在本指南的范圍內(nèi)，但銀行組織使用的更多定性方法（即那些未根據(jù)本指南定義為模型的方法）也應(yīng)接受嚴(yán)格的控制過程。??

3. 如果模型用于生成包含在公共財務(wù)報表中的金額，則對模型不確定性的任何調(diào)整都必須符合公認(rèn)的會計原則。??

4. 見http://www.federalreserve.gov/feedback.cfm。??

TO THE OFFICER IN CHARGE OF SUPERVISION AND APPROPRIATE SUPERVISORY AND EXAMINATION STAFF AT EACH FEDERAL RESERVE BANK

SUBJECT:

Guidance on Model Risk Management

The Federal Reserve and Office of the Comptroller of the Currency (OCC) are issuing the attached Supervisory Guidance on Model Risk Management, which is intended for use by banking organizations and supervisors as they assess organizations’ management of model risk. This guidance should be applied as appropriate to all banking organizations supervised by the Federal Reserve, taking into account each organization’s size, nature, and complexity, as well as the extent and sophistication of its use of models (as defined and discussed below).

Model Risk Management

Banking organizations should be attentive to the possible adverse consequences (including financial loss) of decisions based on models that are incorrect or misused, and should address those consequences through active model risk management. The attachment to this SR letter describes in more detail the key aspects of an effective model risk management framework, including robust model development, implementation, and use; effective validation; and sound governance, policies, and controls.

Previous publications issued by the Federal Reserve and OCC have addressed the use of models, with particular focus on model validation.1 Based on supervisory and industry experience over the past several years, this document expands upon existing guidance—most importantly by broadening the scope to include other key aspects of model risk management.

For the purposes of this document, the term model refers to a quantitative method, system, or approach that applies statistical, economic, financial, or mathematical theories, techniques, and assumptions to process input data into quantitative estimates. Models meeting this definition might be used for analyzing business strategies, informing business decisions, identifying and measuring risks, valuing exposures, instruments or positions, conducting stress testing, assessing adequacy of capital, managing client assets, measuring compliance with internal limits, maintaining the formal control apparatus of the bank, or meeting financial or regulatory reporting requirements and issuing public disclosures. The definition of model also covers quantitative approaches whose inputs are partially or wholly qualitative or based on expert judgment, provided that the output is quantitative in nature.2

The use of models invariably presents model risk, which is the potential for adverse consequences from decisions based on incorrect or misused model outputs and reports. Model risk can lead to financial loss, poor business and strategic decision-making, or damage to a banking organization’s reputation. Model risk occurs primarily for two reasons:? (1) a model may have fundamental errors and produce inaccurate outputs when viewed against its design objective and intended business uses; (2) a model may be used incorrectly or inappropriately or there may be a misunderstanding about its limitations and assumptions. Model risk increases with greater model complexity, higher uncertainty about inputs and assumptions, broader extent of use, and larger potential impact.? Banking organizations should manage model risk both from individual models and in the aggregate.

A guiding principle throughout the guidance is that managing model risk involves "effective challenge" of models:? critical analysis by objective, informed parties that can identify model limitations and produce appropriate changes. Effective challenge depends on a combination of incentives, competence, and influence.

As is generally the case with other risks, materiality is an important consideration in model risk management. If at some banks the use of models is less pervasive and has less impact on their financial condition, then those banks may not need as complex an approach to model risk management in order to meet supervisory expectations. However, where models and model output have a material impact on business decisions, including decisions related to risk management and capital and liquidity planning, and where model failure would have a particularly harmful impact on a bank’s financial condition, a bank’s model risk management framework should be more extensive and rigorous.

Model Development, Implementation, and Use

Model development relies heavily on the experience and judgment of developers, and model risk management should include disciplined model development and implementation processes that are consistent with the situation and goals of the model user and with the banking organization’s policy. A sound development process includes:? a clear statement of purpose to ensure that the model is developed in line with its intended use; sound design, theory, and logic underlying the model; robust model methodologies and processing components; rigorous assessment of data quality and relevance; and appropriate documentation. An integral part of model development is testing, in which the various components of a model and its overall functioning are evaluated to show the model is performing as intended; to demonstrate that it is accurate, robust, and stable; and to evaluate its limitations and assumptions. Importantly, organizations should ensure that the development of the more judgmental and qualitative aspects of their models is also sound.

All models have some degree of uncertainty and inaccuracy because they are by definition imperfect representations of reality. An important outcome of effective model development, implementation, and use is a banking organization’s demonstrated understanding of and accounting for such uncertainty. Accounting for model uncertainty can include applying well-supported, judgmental, “conservative” adjustments to model output, placing less emphasis on a model’s output, or ensuring that a model is only used when supplemented by other models or approaches.3

Model Validation

Model validation is the set of processes and activities intended to verify that models are performing as expected, in line with their design objectives and business uses. Effective validation helps to ensure that models are sound, identifying potential limitations and assumptions and assessing their possible impact. All model components—inputs, processing, outputs, and reports—should be subject to validation; this applies equally to models developed in-house and to those purchased from or developed by vendors or consultants.

Validation involves a degree of independence from model development and use. Generally, validation is done by staff who are not responsible for model development or use and do not have a stake in whether a model is determined to be valid. As a practical matter, some validation work may be most effectively done by model developers and users; it is essential, however, that such validation work be subject to critical review by an independent party, who should conduct additional activities to ensure proper validation. Overall, the quality of the validation process is indicated by critical review by objective, knowledgeable parties and the actions taken to address issues identified by those parties.

Validation activities should continue on an ongoing basis after a model goes into use to track known model limitations and to identify any new ones. Validation is an important check during periods of benign economic and financial conditions, when estimates of risk and potential loss can become overly optimistic and the data at hand may not fully reflect more stressed conditions. Banking organizations should conduct a periodic review—at least annually but more frequently if warranted—of each model to determine whether it is working as intended and if the existing validation activities are sufficient. Key elements of comprehensive validation include:

Evaluation of Conceptual Soundness. This element involves assessing the quality of the model design and construction, as well as review of documentation and empirical evidence supporting the methods used and variables selected for the model. This step in validation should ensure that judgment exercised in model design and construction is well informed, carefully considered, and consistent with published research and with sound industry practice.

Ongoing Monitoring. This step in validation is done to confirm that the model is appropriately implemented and is being used and performing as intended.? It is essential to evaluate whether changes in products, exposures, activities, clients, or market conditions necessitate adjustment, redevelopment, or replacement of the model and to verify that any extension of the model beyond its original scope is valid. Benchmarking can be used in this step to compare a given model’s inputs and outputs to estimates from alternatives.

Outcomes Analysis. This step involves comparing model outputs to corresponding actual outcomes. Back-testing is one form of outcomes analysis that involves the comparison of actual outcomes with model forecasts during a sample time period not used in model development at a frequency that matches the model’s forecast horizon or performance window.

The results of the three core elements of the validation process may reveal significant errors or inaccuracies in model development or outcomes that consistently fall outside the banking organization’s predetermined thresholds of acceptability. In such cases, model adjustment, recalibration, or redevelopment is warranted. At times, banking organizations may have a limited ability to use key model validation tools for various reasons, such as lack of data or of price observability. In those cases, even more attention should be paid to the model’s limitations when considering the appropriateness of model usage, and senior management should be fully informed of those limitations when using the models for decision-making. Generally, senior management should ensure that appropriate mitigating steps are taken in light of identified model limitations, which can include adjustments to model output, restrictions on model use, reliance on other models or approaches, or other compensating controls

Governance, Policies, and Controls

Developing and maintaining strong governance over the model risk management framework is fundamentally important to its effectiveness. Strong governance provides explicit support and structure to risk management functions through policies defining relevant risk management activities, procedures that implement those policies, allocation of resources, and mechanisms for testing that policies and procedures are being carried out as specified. Strong governance also includes documentation of model development and validation that is sufficiently detailed to allow parties unfamiliar with a model to understand how the model operates, as well as its limitations and key assumptions.

Model risk governance is provided at the highest level by the board of directors and senior management when they establish an organization-wide approach to model risk management. Board members should ensure that the level of model risk is within their tolerance. A banking organization’s internal audit function should assess the overall effectiveness of the model risk management framework, including the framework’s ability to address both types of model risk for individual models and in the aggregate. Whenever a banking organization uses external resources for model risk management, the organization should specify the activities to be conducted in a clearly written and agreed-upon scope of work, and those activities should be conducted in accordance with this guidance. Also, organizations should maintain an inventory of models implemented for use, under development for implementation, or recently retired.

All banking organizations should ensure that their internal policies and procedures are consistent with the risk management principles and supervisory expectations contained in this guidance.

Contacts

For questions regarding this guidance, please contact David Palmer, Senior Supervisory Financial Analyst, Risk, at (202) 452-2904; Dwight Smith, Senior Supervisory Financial Analyst, Capital & Regulatory Policy, at (202) 452-2773; or Anna Lee Hewko, Assistant Director, at (202) 530-6260. In addition, questions may be sent via the Board’s public website.4

signed by

Patrick M. Parkinson

Director

Division of Banking

Supervision and Regulation

Attachments:

Model Risk Management Guidance (PDF)

Cross References:

SR 09-1, "Application of the Market Risk Rule in Bank Holding Companies and State Member Banks"

Notes:

For instance, the OCC provided guidance on model risk, focusing on model validation, in OCC 2000-16 (May 30, 2000), other bulletins, and certain subject matter booklets of the Comptroller’s Handbook. The Federal Reserve issued SR Letter 09-01, “Application of the Market Risk Rule in Bank Holding Companies and State Member Banks,” which highlights various concepts pertinent to model risk management, including standards for validation and review, model validation documentation, and back-testing. The Federal Reserve’s Trading and Capital-Markets Activities Manual also discusses validation and model risk management. In addition, the advanced-approaches risk-based capital rules (12 CFR 3, Appendix C; 12 CFR 208, Appendix F; and 12 CFR 225, Appendix G) contain explicit validation requirements for subject banking organizations.? Return to text

While outside the scope of this guidance, more qualitative approaches used by banking organizations—i.e., those not defined as models according to this guidance—should also be subject to a rigorous control process.? Return to text

To the extent that models are used to generate amounts included in public financial statements, any adjustments for model uncertainty must comply with generally accepted accounting principles.? Return to text

See http://www.federalreserve.gov/feedback.cfm.??