[Request] Expose TypeScript types.

[gluons]

Is your request related to a specific problem you're having?
Sometime I want to use some type from Highlight.js internal types. e.g. Language, LanguageFn.
But I can't use it now.

The solution you'd prefer / feature you'd like to see added...
Expose all internal TypeScript types from highlight.js.
So I can use it for more typing.

Any alternative solutions you considered...
No.

Additional context...
Types should be able to import like this.

import { LanguageFn, Language } from 'highlight.js';

[joshgoebel]

I really am not familiar with the type publishing types side of this equation. Do you know what that would look like? If I just add export to the types will that magically work?

There is also this which might or might not be related. #2682

---

We only use TypeScript for development listing but if someone knows how to improve this without making it way more complicated I'm all ears.

[joshgoebel]

Also, please provide a list. We're not going to export purely internal types because we do not wish to encourage their use, as anything that is not public API is subject to change at any time - and if you're consuming it then you do so at your own risk.

Things I see that might be useful for plugin/grammar authors (at a quick glance):

• HLJSOptions
• Language
• LanguageFn
• Mode
• ModeCallback

[joshgoebel]

Though if we export them, do we then need to namespace them all? If so can we have them namespaces only externally? Forcing a namespace we don't need internally is just ugh.

These are the things I don't know and it would be great to have a TypeScript expert to help with.

[gluons]

The types that we (any plugin developers) may face are types that used in public functions.
I think it's here.

• Mode
• HighlightResult (From highlight)
• AutoHighlightResult (From highlightAuto)
• HLJSOptions
• LanguageFn
• Language
• HLJSPlugin (From addPlugin)
• VuePlugin (From vuePlugin)

May need?

• Emitter (From HighlightResult.emitter)
• CompiledMode (From HighlightResult.top)
• illegalData (From HighlightResult.illegalBy)

[gluons]

Writing type declaration is not easy for me. 😅
I usually write code with TypeScript and let it generate types for me.
Hope someone can help. 🥺

[joshgoebel]

VuePlugin (From vuePlugin)

This is just the type we use for our VuePlugin... I'm not sure it's something we should expose. Someone writing their own Vue plugin probably wants to declare their own type or use a built-in one in Vue maybe.

HighlightResult (From highlight)
AutoHighlightResult (From highlightAuto)

Don't you get these for free just by using the public API? Why might someone need to declare one of these from scratch? Perhaps one or two of the plugin APIs allow the result to be replaced? I'll check. If so then this makes sense.

Emitter (From HighlightResult.emitter)

Psuedo-private/beta... probably going to keep internal.

CompiledMode (From HighlightResult.top)

CompiledModes are technically internal and subject to change at any moment. Only compiler extensions should care about this and for now our guidance is those aren't really intended for 3rd party use. Perhaps we need a way to flag things on result that are more for internal/debugging uses... (which is the intention of top) Code that depends on it's internal structure is likely to be fragile.

illegalData (From HighlightResult.illegalBy)

Same thought: debugging/internal usage. Open to suggestions on how to flag these in the future, perhaps they should change to _ prefix or some such?

[gluons]

VuePlugin (From vuePlugin)

Yes. This type seems unnecessary.
Vue already provides PluginObject for plugin object ({ install: (....) => {} }).

Emitter, CompiledMode, illegalData

Agree. Internal uses. May be unnecessary to use from public API.

HighlightResult, AutoHighlightResult

May be used in function that accept result from Highlight.js.
e.g.

function processSomeResult(result: HighlightResult) {
	// ...
}

[joshgoebel]

Can you test our project from a checkout of Highlight.js from Git? And just add export to a few lines and see if that magically does what you hope?

[gluons]

From master, right?

[joshgoebel]

Yes.

[gluons]

My edit

/* eslint-disable no-unused-vars */
/* eslint-disable no-use-before-define */
// For TS consumers who use Node and don't have dom in their tsconfig lib, import the necessary types here.
/// <reference lib="dom" />

/* Public API */

// eslint-disable-next-line
declare const hljs : HLJSApi;

type HLJSApi = PublicApi & ModesAPI

interface VuePlugin {
    install: (vue: any) => void
}

interface PublicApi {
    highlight: (languageName: string, code: string, ignoreIllegals?: boolean, continuation?: Mode) => HighlightResult
    highlightAuto: (code: string, languageSubset?: string[]) => AutoHighlightResult
    fixMarkup: (html: string) => string
    highlightBlock: (element: HTMLElement) => void
    configure: (options: Partial<HLJSOptions>) => void
    initHighlighting: () => void
    initHighlightingOnLoad: () => void
    registerLanguage: (languageName: string, language: LanguageFn) => void
    listLanguages: () => string[]
    registerAliases: (aliasList: string | string[], { languageName } : {languageName: string}) => void
    getLanguage: (languageName: string) => Language | undefined
    requireLanguage: (languageName: string) => Language | never
    autoDetection: (languageName: string) => boolean
    inherit: <T>(original: T, ...args: Record<string, any>[]) => T
    addPlugin: (plugin: HLJSPlugin) => void
    debugMode: () => void
    safeMode: () => void
    versionString: string
    vuePlugin: () => VuePlugin
}

interface ModesAPI {
    SHEBANG: (mode?: Partial<Mode> & {binary?: string | RegExp}) => Mode
    BACKSLASH_ESCAPE: Mode
    QUOTE_STRING_MODE: Mode
    APOS_STRING_MODE: Mode
    PHRASAL_WORDS_MODE: Mode
    COMMENT: (begin: string | RegExp, end: string | RegExp, modeOpts?: Mode | {}) => Mode
    C_LINE_COMMENT_MODE: Mode
    C_BLOCK_COMMENT_MODE: Mode
    HASH_COMMENT_MODE: Mode
    NUMBER_MODE: Mode
    C_NUMBER_MODE: Mode
    BINARY_NUMBER_MODE: Mode
    CSS_NUMBER_MODE: Mode
    REGEXP_MODE: Mode
    TITLE_MODE: Mode
    UNDERSCORE_TITLE_MODE: Mode
    METHOD_GUARD: Mode
    END_SAME_AS_BEGIN: (mode: Mode) => Mode
    // built in regex
    IDENT_RE: string
    UNDERSCORE_IDENT_RE: string
    NUMBER_RE: string
    C_NUMBER_RE: string
    BINARY_NUMBER_RE: string
    RE_STARTERS_RE: string
}

-type LanguageFn = (hljs?: HLJSApi) => Language
+export type LanguageFn = (hljs?: HLJSApi) => Language
type CompilerExt = (mode: Mode, parent: Mode | Language | null) => void

-interface HighlightResult {
+export interface HighlightResult {
    relevance : number
    value : string
    language? : string
    emitter : Emitter
    illegal : boolean
    top? : Language | CompiledMode
    illegalBy? : illegalData
    sofar? : string
    errorRaised? : Error
    // * for auto-highlight
    second_best? : Omit<HighlightResult, 'second_best'>
    code?: string
}
-interface AutoHighlightResult extends HighlightResult {}
+export interface AutoHighlightResult extends HighlightResult {}

interface illegalData {
    msg: string
    context: string
    mode: CompiledMode
}

type BeforeHighlightContext = {
  code: string,
  language: string,
  result?: HighlightResult
}
type PluginEvent = keyof HLJSPlugin;
-type HLJSPlugin = {
+export type HLJSPlugin = {
    'after:highlight'?: (result: HighlightResult) => void,
    'before:highlight'?: (context: BeforeHighlightContext) => void,
    'after:highlightBlock'?: (data: { result: HighlightResult}) => void,
    'before:highlightBlock'?: (data: { block: Element, language: string}) => void,
}

interface EmitterConstructor {
    new (opts: any): Emitter
}

-interface HLJSOptions {
+export interface HLJSOptions {
   noHighlightRe: RegExp
   languageDetectRe: RegExp
   classPrefix: string
   tabReplace?: string
   useBR: boolean
   languages?: string[]
   __emitter: EmitterConstructor
}

interface CallbackResponse {
    data: Record<string, any>
    ignoreMatch: () => void
}

/************
 PRIVATE API
 ************/

/* for jsdoc annotations in the JS source files */

type AnnotatedError = Error & {mode?: Mode | Language, languageName?: string, badRule?: Mode}

type ModeCallback = (match: RegExpMatchArray, response: CallbackResponse) => void
type HighlightedHTMLElement = HTMLElement & {result?: object, second_best?: object, parentNode: HTMLElement}
type EnhancedMatch = RegExpMatchArray & {rule: CompiledMode, type: MatchType}
type MatchType = "begin" | "end" | "illegal"

 interface Emitter {
    addKeyword(text: string, kind: string): void
    addText(text: string): void
    toHTML(): string
    finalize(): void
    closeAllNodes(): void
    openNode(kind: string): void
    closeNode(): void
    addSublanguage(emitter: Emitter, subLanguageName: string): void
 }

/* modes */

 interface ModeCallbacks {
     "on:end"?: Function,
     "on:begin"?: ModeCallback
 }

-interface Mode extends ModeCallbacks, ModeDetails {
+export interface Mode extends ModeCallbacks, ModeDetails {

}

interface LanguageDetail {
    name?: string
    rawDefinition?: () => Language
    aliases?: string[]
    disableAutodetect?: boolean
    contains: (Mode)[]
    case_insensitive?: boolean
    keywords?: Record<string, any> | string
    compiled?: boolean,
    exports?: any,
    classNameAliases?: Record<string, string>
    compilerExtensions?: CompilerExt[]
    supersetOf?: string
}

-type Language = LanguageDetail & Partial<Mode>
+export type Language = LanguageDetail & Partial<Mode>

interface CompiledLanguage extends LanguageDetail, CompiledMode {
    compiled: true
    contains: CompiledMode[]
    keywords: Record<string, any>
}

type KeywordData = [string, number];
type KeywordDict = Record<string, KeywordData>

type CompiledMode = Omit<Mode, 'contains'> &
    {
        contains: CompiledMode[]
        keywords: KeywordDict
        data: Record<string, any>
        terminatorEnd: string
        keywordPatternRe: RegExp
        beginRe: RegExp
        endRe: RegExp
        illegalRe: RegExp
        matcher: any
        compiled: true
        starts?: CompiledMode
        parent?: CompiledMode
    }

interface ModeDetails {
    begin?: RegExp | string
    match?: RegExp | string
    end?: RegExp | string
    className?: string
    contains?: ("self" | Mode)[]
    endsParent?: boolean
    endsWithParent?: boolean
    endSameAsBegin?: boolean
    skip?: boolean
    excludeBegin?: boolean
    excludeEnd?: boolean
    returnBegin?: boolean
    returnEnd?: boolean
    __beforeBegin?: Function
    parent?: Mode
    starts?:Mode
    lexemes?: string | RegExp
    keywords?: Record<string, any> | string
    beginKeywords?: string
    relevance?: number
    illegal?: string | RegExp | Array<string | RegExp>
    variants?: Mode[]
    cachedVariants?: Mode[]
    // parsed
    subLanguage?: string | string[]
    compiled?: boolean
    label?: string
}

// deprecated API since v10
// declare module 'highlight.js/lib/highlight.js';

declare module 'highlight.js' {
    export = hljs;
}

declare module 'highlight.js/lib/core' {
    export = hljs;
}

declare module 'highlight.js/lib/core.js' {
    export = hljs;
}

declare module 'highlight.js/lib/languages/*' {
    export default function(hljs?: HLJSApi): LanguageDetail;
}

---

No luck.
It seems to work on main module (import { ... } from 'highlight.js') but it breaks /lib/core module. 😞

[joshgoebel]

I don't think this can work because we purposely use default export for highlight.js. IE:

import hljs from 'highlight.js';

This is fundamentally incompatible with named exports. I'd be curious to know how other libs do this.

[peterblazejewicz]

@joshgoebel

the index.d.ts could export types/interfaces, it does not impact es6 exports.
you could, for example:

declare const hljs : HLJSApi;

export type HLJSApi = PublicApi & ModesAPI
..
..
..
export default hljs;

declare module 'highlight.js/lib/core' {
    export = hljs;
}

declare module 'highlight.js/lib/core.js' {
    export = hljs;
}

declare module 'highlight.js/lib/languages/*' {
    export default function(hljs?: HLJSApi): LanguageDetail;
}

and in other file:

/**
 * @typedef {import('../types').HLJSPlugin} HLJSPlugin
 * @typedef {import('../types').Language} Language
 * @typedef {import('../types').HLJSApi} HLJSApi
 * @typedef {import('../types').HLJSOptions} HLJSOptions
 */

or via regular DT resolution.
the point is, the file has nothing exported, IMO
hth

[joshgoebel]

Anytime an export is added it breaks the declare modules with:

module "highlight.js/lib/core"
Invalid module name in augmentation. Module 'highlight.js/lib/core' resolves to an untyped module at '/Users/jgoebel/work/highlight.js/build/lib/core.js', which cannot be augmented.ts(2665)

Though it only seems to bother these two, which is so confusing:

declare module 'highlight.js/lib/core' {
    export = hljs;
}

declare module 'highlight.js/lib/core.js' {
    export = hljs;
}

[peterblazejewicz]

IMO you don't have there anything, there is nothing to augment: (there is no `highlight.js/lib/core'):
https://github.com/highlightjs/highlight.js/tree/main/src/lib

[joshgoebel]

Ok, maybe that's an error that doesn't matter... lib/core exists in our final builds, but not our source tree.

But now if I add exports in the main index.d.ts body:

export type Language = LanguageDetail & Partial<Mode>

It still can't be imported later:

b.ts:1:22 - error TS2305: Module '"highlight.js"' has no exported member 'Language'.

1 import { LanguageFn, Language } from 'highlight.js';

I assume it's because it's not mentioned in our declare module:

declare module 'highlight.js' {
    export = hljs;
}

Which leads me back to... how can we tell TS that we're exporting the object (hljs) but also export types? Or perhaps that isn't possible because our source is JS, not TS?

[peterblazejewicz]

b.ts:1:22 - error TS2305: Module '"highlight.js"' has no exported member 'Language'.

one needs to add explicit export here:

export type Language = ...

your current .d.ts has no explicit exports, only single top level object export. It won't auto export types, unless you introduce a namespace afaik.

[joshgoebel]

one needs to add explicit export here:

I added that locally, it makes no difference. Actually it adds the following errors:

error TS2666: Exports and export assignments are not permitted in module augmentations.
Could not find a declaration file for module 'highlight.js/lib/core'.

I assume the declare module is overriding everything.

And if I even try to put them inside the same declare module:

An export assignment cannot be used in a module with other exported elements.ts(2309)

[joshgoebel]

It won't auto export types, unless you introduce a namespace afaik.

I don't mind flagging what needs to be exported myself, I just have no idea how to do it.

[peterblazejewicz]

is this to be used within project itself, or for consumption by users? (kind of @types/higlight.js)

[joshgoebel]

Both. I use it for with-in editor Linting but it should also work in place of @types/higlight.js (all @types/highlight.js does now is re-export us).

I don't have to import the types though within the project. I presume they "just work" because of the package.json types.

[peterblazejewicz]

so your project looks different once built and published. there are lib/core/index.js and lib/core.js. The old DT would also incorporate 'index.d.ts' on the root level to just re-export main objet and - important - to define and export shared types. Then within lib declarations people are just importing types from root definition for consumption. I'll check this tmrw. Also, you could also raise that topic on the TypeScirpt discord. A lot of people knows higliht.js: https://discord.gg/UHDsndNs

[joshgoebel]

there are lib/core/index.js...

No, but there is lib/core.js and lib/index.js and soon lib/common.js. Which all just include a different # of languages (0, common, all) and re-export the main object.

also incorporate 'index.d.ts' on the root level

Are you saying the file must be at the top of the package? Is this not what package.json types is for to allow placement of this main file anywhere?

Most of what you said is greek to me, sorry. I really need a specific example or PR I think.

[joshgoebel]

@gluons Can you see if this helps?

#3073

[peterblazejewicz]

@joshgoebel #3075 sample for showing the possible iteration.
This one is based on the consumption in local project (TS/TSC)

[joshgoebel]

That's about the same as what I did in #3073 but using multiple files, right?

[peterblazejewicz]

Yes, + named export for global, non module, usage (if you care about those users)

[joshgoebel]

@gluons Ping. Could you test the PR?

[gluons]

OK. I'll try at weekend.

[gluons]

@joshgoebel I've tested #3073 with fresh project.
Types can be imported. it works very well.
But highlight.js/lib/core can't be imported.

[joshgoebel]

Can you zip your fresh project or push it to a GitHub repo so I can pull it as-is?

[gluons]

@joshgoebel Here. I used local dependency that clone from #3073.

hljs-test.zip

---

I don't know why it doesn't work. I've seen src/core.d.ts from #3073.
I've tried to run npm run build in cloned repo but it doesn't generate lib directory.
Is it unusual? (Or I made some mistake?)
I'm running on Windows 10.

[joshgoebel]

I've tried to run npm run build in cloned repo

The build artifacts are in build.

[joshgoebel]

I updated the relative path in package.json (my paths are deeper) but it builds fine for me with 0 errors:

tsc src/*.ts

tsc works also but throws a bunch of iterator errors with ESNext that have nothing to do with us.

I'm thinking something is subtly wrong with your setup or you aren't building the latest #3073?

[gluons]

The build artifacts are in build.

OK. Maybe my mistake.

---

I've copied all build artifacts into new highlight.js directory as local dependency.
It works very. Thank you.

[joshgoebel]

Awesome!