Test plan for "top-level statements" feature

[jcouv]

Championed issue: dotnet/csharplang#2765
Speclet: https://github.com/dotnet/csharplang/blob/master/proposals/top-level-statements.md

Compiler:

• Language version errors (LanguageVersion_01, Scope_03, Scope_06)
• Top-level statements across multiple compilation units (Simple_03, Simple_05)
  • Locals and local functions defined and used in distinct compilation units (Simple_06_02, Simple_07, LocalUsedBeforeDeclaration_01, LocalUsedBeforeDeclaration_02, LocalUsedBeforeDeclaration_03, LocalUsedBeforeDeclaration_04, NullableRewrite_01, Scope_02, Scope_03, Scope_05, Scope_06, Scope_16, Scope_18, Scope_20, Scope_22, Scope_24)
  • Locals re-defined in distinct compilation units (LocalDeclarationStatement_07)
• Top-level statements before using declarations and after namespaces and type declarations
• Statements as top-level statements including:
  • ;, { }
  • labels, goto, break, continue, return
  • yield return, yield break, await
• Member declarations as top-level statements including: fields, events, methods, properties, indexers, operators, explicit implementations
• $Program.$Main has expected signature:
  • Parameter: string[] args
  • Return type: void, int, Task, or Task<int> depending on explicit return statements and use of await
  • Use-site errors for types used in $Program and $Main declarations
• Command-line arguments are available from string[] args
• Top-level statements combined with explicit declaration of valid Main entry point
  • Consider -main:<type> option
• Top-level statements when compiling a library, a net module, a .winmd
• SymbolDisplay and diagnostics containing $Program and $Main
• Variable scope:
  • Locals, local functions, and args parameter names used and redefined in top-level statements and local functions
  • Locals, local functions, and args parameter names used and redefined in using statements and type declarations, in same and distinct compilation units
• All valid and invalid modifiers for locals and local functions
• Attributes on locals and local functions
• Capturing of locals and args parameter in local functions
• Emit and execution of top-level statements and local functions
• Local function with extension method this argument
• Flow analysis of top-level statements
  • Definite assignment for single and multiple compilation units
  • Unreachable code
• Nullable analysis for single and multiple compilation units
• Unsafe code in top-level statements with/without unsafe block, with/without -unsafe command-line option
  • LDM question: should unsafe code be allowed without unsafe block?
• $Main is used for [CallerMemberName] arguments
• Explicit return statement in some branches only: if (expr) { return 1; }
• Locals named await and async
• Add "Top-level statements" to compiler test plan
• ref locals with await
• ref escaping (e.g. storing Span<int> from a stackalloc local in a field)

Public API:

• ISymbol instances for $Program and $Main
• IOperation for single and multiple compilation units
• CFG for locals and local functions in single and multiple compilation units
• Analyzer actions for single an multiple compilation units
• LookupSymbols and LookupNames:
  • For single and multiple compilation units
  • In using statements before top-level statements
  • In type declarations after top-level statements

Productivity:

• Completion on args
• QuickInfo and GoToDefinition on args
• Semantic classification of args (maybe should be colored similarly to value in property setter)

Debugging (compiler):

• $Main is correctly marked as an entry point in the PDB
• F10 on a console app project with top-level code, the debugger stops at the first statement of the top-level code (for both async and sync code)
• Expression Evaluator works in the context of top-level code (usings, args and locals are in scope when evaluating expression in context of top-level code)
• Edit and Continue - emit delta works (adding a top-level type, top-level local function, updating body of a top-level local function, modifying statements in top-level code) (EnC is currently not working in a compilation with top-level statements A change in a top-level statement or an addition of a new top-level statement is not picked up by ENC #44196)

Debugging (IDE):

• Edit and Continue works end-to-end (EnC is currently not working in a compilation with top-level statements A change in a top-level statement or an addition of a new top-level statement is not picked up by ENC #44196)

FYI @AlekseyTs

[AlekseyTs]

Core Scenarios in Editing, Navigating, and Viewing

Category	Feature/Description	C# Signoff/Notes	VB Signoff/Notes	F# Signoff/Notes
Enable/Disable	⏩ Feature Flags To completely enables/disable new compiler features in the compiler & IDE	✅ Language version upgrade is offered		N/A
Typing	⏩ General Typing - Type and paste new constructs - Nothing interferes with verbatim typing	🆗 Completion interferes. For example, types are not offered at the beginning of a local declaration, int followed by a space completes as interface.
	🔍 ⏩ Completion - Typing new keyword/construct names - Dotting off of new constructs - Matching part of the identifier is highlighted (including word prefix matches) [Visual Studio 2015 Update 1] - Target type preselection [Visual Studio 2017] IntelliSense filtering [Visual Studio 2017]	🆗 No completion at the beginning of the line following System.Console.WriteLine();, but there is a completion after a for with a block body.
	⏩ Formatting - Spacing in and around new constructs - Spacing options - Format Document command Tools > Options settings should be respected	✅
	⏩ Automatic Brace Completion (C# only) - Auto-insert close brace - Shift+Enter commit of IntelliSense and any pending brace completion sessions (Known issue: #18065)	✅		N/A
	⏩ Indentation - Typing Enter in an unfinished statement indents the next line	✅
Navigating	🔍 ⏩ Go To Definition - F12 from callsites to definition - Ctrl+click [Visual Studio 2017 version 15.4]	✅
	⏩ Go To Implementation - Ctrl+F12 to jump from virtual members to their implementations - Jump from inheritable types to their implementations	N/A		N/A
	🔍 ⏩ Find All References - Lists references to a symbol in "Find Symbol Results" window - Shows results in hierarchy grouped by definition [Visual Studio 2015] - Results should be groupable/filterable/classified appropriately [Visual Studio 2017] - Find All References on literals [Visual Studio 2017 version 15.3]	✅
Viewing	🔍 ⏩ Colorization - Keywords, literals, and identifiers colored appropriately in code - Colors should theme appropriately - The "Blue Theme (Additional Contrast)" should have sufficiently dark colors	✅
	⏩ Error Squiggles - Squiggles appear as expected on reasonable spans	✅

[AlekseyTs]

Code Transformations

Category	Feature/Description	C# Signoff/Notes	VB Signoff/Notes	F# Signoff/Notes
Refactoring with UI	🔍 Inline Rename (with UI) - Dashboard shows correct information - Highlighted spans are updated appropriately - Rename operation updates the correct set of symbols	✅
	🔍 Change Signature (with UI) - Updates all direct & cascaded definitions/callsites - Shows appropriate signature & parameter previews in UI - Reorder and Remove in the same session [Visual Studio 2015]	❓ It looks like the feature is completely broken in the current build. See #44198.		N/A
	🔍 Extract Interface (with UI) - Generated Interface has expected shape - UI shows appropriate method previews	N/A		N/A
	Generate Type (with UI) - Dialog gives all valid options	🚧 The type is generated in between of the top-level statements, which is a syntax error.		N/A
	Generate Overrides [Visual Studio 2017 version 15.3]	N/A		N/A
Refactorings	Rename Tracking - Tracking span tracks & dismisses as expected - Invokable from references [Visual Studio 2015]	✅		N/A
	🔍 Extract Method - Extracted method has the expected signature - All arguments/return values handled correctly - Extracted code block is reasonable - Automatically starts Inline Rename	🆗 Not offered. Also, sometimes a notification about crash pops up #44260		N/A
	Introduce Variable - Introduced variable has the expected signature and initializer expression - "Introduce for All" correctly finds dupes	🚧 Not offered		N/A
	Inline Temporary Variable - Inlined values are appropriately expanded/reduced	🆗 Asserts, then crashes #44263		N/A
	Organize Usings - Honors "Place 'System' namespace first" option	N/A		N/A
	Convert Get* Methods to Properties Add Description	N/A		N/A
	🔍 Encapsulate Field - Select a field and convert it to a property backed by the field - Try selecting multiple fields at once	N/A		N/A
	Modifier Ordering [Visual Studio 2017 version 15.5]	N/A		N/A
	Convert ReferenceEquals to is null [Visual Studio 2017 version 15.5]	✅		N/A
	Add Missing Modifiers [Visual Studio 2017 version 15.5]	N/A		N/A
	Convert Lambda to Local Function [Visual Studio 2017 version 15.5]	❓ Not sure how to invoke		N/A
	Move Declaration Near Reference [Visual Studio 2017 version 15.5]	🚧 Not offered		N/A
	Introduce Pattern Matching [Visual Studio 2017 version 15.5]			N/A
	Simplify Inferred Tuple Names [Visual Studio 2017 version 15.5]	Requires C# 7.1 or greater		N/A
	Convert keyword and symbol references to doc comment tags [Visual Studio 2017 version 15.5]	N/A		N/A
Fixes	Add Using - Triggers on appropriate constructs - Including NuGet references - Including Referenced Assemblies - Includes spelling fixes	✅
	Generate Local - Select an expression and introduce a local variable to represent it - This should start an Inline Rename session	🚧 Not offered		N/A
	Generate Field - Select an expression and introduce a field to represent it - This should start an Inline Rename session	🚧 Offered on a new name, but mentions an unspeakable type $Program. Should not be offering that.		N/A
	Generate Method/Constructor - Call a nonexistent method or constructor to generate it from its usage - Generated method has the expected signature and accessibility - Add parameter to existing constructor from callsite [Visual Studio 2017 version 15.3]	🚧 Generating method/constructor in a class works. Adding argument to a method in a class adds a parameter to the method. Adding argument to constructor, adds new constructor. For a new local function invocation, there is an offer to add a method into an unspeakable $Program type. Using that actually generates a method that is a valid local function of the right shape. Should offer to generate a local function instead. 🆗 Adding an argument to a local function invocation and using "Add parameter" fix, crashes CSharpAddParameterCodeFixProvider (#44271).		N/A
	Generate Constructor from members - Select fields/properties to generate a constructor accepting corresponding arguments - Generated constructor has the expected signature and accessibility	N/A		N/A
	Implement Interface - Only missing methods added - All added methods have the expected signature and accessibility	N/A		N/A
	Implement IDisposable - Implement IDisposable and you should see a large block of code properly implementing that particular interface	N/A		N/A
	Implement Abstract Class - Inherit from an abstract class, and you should be able to auto-generate all of the missing members	N/A		N/A
	Remove Unused Variables [Visual Studio 2017 version 15.3]	🆗 #44272, #44273		N/A
	Remove Unused Usings	N/A
	Sort Usings	N/A		N/A
	Convert Get Methods to Properties - Name a method GetStuff and convert it to a property called Stuff	/N/A		N/A
	Make Method Async/Sync - Add an await to a synchronous method, you should be offered to add the async keyword - Remove all await keywords from an async method, you should be offered to remove the async keyword	✅ Works for top level local functions.		N/A
	Use Object Initializer Over Property Assignment - Create a new instance of a type and then assign each of its properties on subsequent lines - You should be offered to convert that to an object initializer	🚧 Not offered.		N/A
	Insert Digit Separators [Visual Studio 2017 version 15.3]	✅		N/A
Code Gen	🔍 Snippets - Tab completion, presence in completion list - Insertion via Snippet Picker UI (Ctrl + K, Ctrl + X) or (Ctrl + K, Ctrl + S) - (VB) Snippet Picker UI via ?<Tab> - (VB) Special snippet completion list (p?<esc><tab>)	🚧 It looks like it is working, unless completion isn't offered completely		N/A
	Event Hookup on Tab (C# only) - Type "+=" after an event name and QuickInfo shows - Invoking should pick good name & launch Inline Rename	🚧 Offered, but doesn't work after pressing Tab	N/A	N/A
	End Construct Generation (VB only) - Type Sub Test() and hit enter, the End Sub should be generated automatically	N/A		N/A
	Automatic End Construct Update (VB only) - Type Sub Test() and End Sub, changing Sub to Function in either one should update the other	N/A		N/A
	Spell checking - Type a name that's close to a variable name but off by a character or two - Lightbulb should have option to fix up the spelling - Includes type names that will require a using	🚧 Not offered.		N/A
	Move type to file - Lightbulb to move type to another file when the type name doesn't match the filename - Option to change the file name if the type doesn't match the file name	N/A		N/A
	Convert between properties and Get methods - Offers to change a method named GetStuff to a property named Stuff	N/A		N/A
	Convert auto property to full property [Visual Studio 2017 version 15.5]	N/A		N/A
	Add missing cases Use a switch on a strict subset of an Enum's members - It should offer to generate the rest of the cases	✅		N/A
	Add null checks for parameters [Visual Studio 2017 version 15.3]	N/A		N/A
	Change base for numeric literals [Visual Studio 2017 version 15.3]	✅		N/A
	Convert if to switch [Visual Studio 2017 version 15.3]	🚧 #44278		N/A
	Resolve git merge conflicts [Visual Studio 2017 version 15.3]	N/A		N/A
	Add argument name [Visual Studio 2017 version 15.3	✅		N/A
	Fade and remove unreachable code [Visual Studio 2017 version 15.5]	✅		N/A
	Add missing file banner [Visual Studio 2017 version 15.5]	N/A		N/A
	Deconstruct tuple declaration [Visual Studio 2017 version 15.4]	❓ Not sure what this stands for.		N/A

[AlekseyTs]

IDE Features

Category	Feature/Description	C# Signoff/Notes	VB Signoff/Notes	F# Signoff/Notes
General	Signature Help - Overloads shown with appropriate, colorized signature	✅
	Quick Info - Hover on identifiers - On completion list items	✅
	🔍 Outlining - Make sure outlining is enabled under options - Define a method and expect to see a collapsible region around the method definition - Make sure collapse and expand work	✅
	Brace Matching (C# only) - Highlights matching brace token, if caret is on an open or close brace. - Hovering on a close brace shows the code around the open brace	✅	N/A	N/A
	Highlight References - Ensure "Highlight references to symbol under cursor" is enabled in options - If caret is on an identifier, all references to that identifier in the active document should be highlighted - We also show related keywords, so placing the caret on a return should show the other returns, if should show elses, try shows catches and finallys, etc. - Should be able to navigate between highlighted references with Ctrl+Shift+Up/Down	✅
	🔍 Peek Press Alt + F12 after placing the cursor on a predefined Type or predefined member and expect to see to a temporary window showing the appropriate definition	✅
	🔍 Navigation Bars - Open some existing source files, make sure you can navigate around the file choosing classes or methods. - Switch between project contexts - In VB, the NavBar can do code generation for events and for New/Finalize methods	🚧 While in top-level statement, the navigation bar shows that we are in the type that follows all top level statements, shows its members as well.
	Metadata As Source - Press F12 on a predefined type and expect the cursor to move the predefined type definition inside a Metadata-As-Source Generated document. - Expect to see the xml doc comments collapsed above the method.	✅		N/A
	Navigate to Decompiled Source [Visual Studio 2017 version 15.6 Preview 2] - Set the option at Tools -> Options -> Text Editor -> C# -> Advanced -> Enable navigation to decompiled sources (experimental) - F12 on a type defined in metadata - You should see decompiled method bodies, not just declarations			N/A
	🔍 Navigate To - Place caret on a user defined Type reference and press "ctrl + ," - This should list the User Defined Type in the drop down on the Upper Right corner of the editor and selecting this item will move the cursor to the User Reference Definition - Filters per kind of symbol [Visual Studio 2017] - Results should be sorted as follows: [Visual Studio 2017 version 15.6 Preview 4] Expand for details TODO	✅
	Go to Next/Previous Method - Edit.NextMethod and Edit.PreviousMethod should work - You may need to set up keyboard bindings for these commands under Tools > Options > Environment > Keyboard	N/A		N/A
	Solution Explorer Pivots - Define a Type and some members in a sample Document. - Expand the Document listed in the Solution Explorer window and expect to see the Type and Members defined - Right-click types and try Base Types / Derived Types / Is Used By / Implements - Right-click methods and try Calls / Is Called By / Is Used By / Implements	✅		N/A
	Call Hierarchy - Place the caret over a method, right click & select View Hierarchy - This should open the "Call Hierarchy window" listing the methods that call the original method and also the callsites within each calling method.	🚧 The hierarchy mentions unspeakable names "$Program" and "$Main"		N/A
	🔍 Code Lens - Make sure Code Lens is enabled from the options. Look for an adornment on top of each method declaration with lists the number of references for that method, last time someone modified the method, who modified the method and other information	✅		N/A
	Project System - Open/close a project - Add/remove references - Unload/reload projects - Source Control integration (adding references checks out projects, etc.)	N/A
	Debugger IntelliSense - Hit a breakpoint (or step) and verify that there's IntelliSense in the Immediate Window (C#, VB) - Type an expression, and hit enter. Verify it's evaluated. Type another expression. IntelliSense should still work. - (VB) there should be IntelliSense if you type "?" followed by an expression (eg, the text of the line in the window is "?foo") - Hit a breakpoint (or step) and verify that there's IntelliSense in the Watch Window - Hit a breakpoint (or step) and verify that there's IntelliSense in the Quick Watch Window - Verify intellisense in the Conditional Breakpoint view - Verify each of the above scenarios after hitting f5 and hitting another breakpoint, and after stepping	✅
	Breakpoint Spans - The span highlighted when a breakpoint is set should be logical	✅
	Code Model / Class Designer - Right click a file in Solution Explorer & choose View Class Diagram. - This shows the "Class Details" window where you can add/remove members, change their type/accessibility/name, parameter info, etc.			N/A
	Object Browser / Class View - Open object browser and classview and verify that project contents are represented live. - Should be able to invoke find all references, navigate to the definition, and do searches.			N/A
	Lightbulb - Should work with Ctrl+. and the right-click menu - Should include a diff view - Should include options to fix all in Document/Project/Solution, and to Preview Changes Varying icons for lightbulb items [Visual Studio 2017 version 15.4]	✅
	Line Separators - Turn the option on under Tools > Options - Ensure there's a line between methods.	N/A		N/A
	Indent Guides - Vertical dotted lines appear between braces for declaration-level things - Hovering on the line shows context	✅
Code Style	Naming Rules			N/A
	Use this./me.			N/A
	Use predefined type			N/A
	Prefer object/collection initializer			N/A
	Prefer explicit tuple name			N/A
	Prefer coalesce expression over null check			N/A
	Prefer null propagation over null check			N/A
	var preferences (C# only)		N/A	N/A
	Prefer braces (C# only)		N/A	N/A
	Prefer pattern matching over is/as checks (C# only)		N/A	N/A
	Use expression body (C# only)		N/A	N/A
	Prefer inlined variable declaration (C# only)		N/A	N/A
	Prefer throw expression (C# only)		N/A	N/A
	Prefer conditional delegate call (C# only)		N/A	N/A

[tmat]

Added checkboxes for debugging

[jcouv]

I think we're past the statute of limitations for this one ;-)
I scanned through the boxes left unchecked and feel comfortable closing this test plan.