Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Contribute to GitLab
Sign in / Register
Toggle navigation
C
CharIP-Electron
Project
Project
Details
Activity
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
0
Issues
0
List
Board
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
ali
CharIP-Electron
Commits
fd1558a6
Commit
fd1558a6
authored
Nov 28, 2023
by
ali
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
feat: TTS 接口接入,完成各项功能支持配置功能
parent
90ce5548
Expand all
Hide whitespace changes
Inline
Side-by-side
Showing
23 changed files
with
1198 additions
and
1002 deletions
+1198
-1002
settings.json
.vscode/settings.json
+1
-1
README.md
README.md
+1
-1
config.js
buildAssets/builder/config.js
+1
-1
IPCs.ts
src/main/IPCs.ts
+44
-29
DefaultLayout.vue
src/renderer/components/layout/DefaultLayout.vue
+4
-4
HeaderLayout.vue
src/renderer/components/layout/HeaderLayout.vue
+28
-39
main.ts
src/renderer/main.ts
+2
-2
index.ts
src/renderer/plugins/asr/index.ts
+1
-1
interfaces.d.ts
src/renderer/plugins/asr/vosk/interfaces.d.ts
+76
-60
model.d.ts
src/renderer/plugins/asr/vosk/model.d.ts
+50
-39
logging.d.ts
src/renderer/plugins/asr/vosk/utils/logging.d.ts
+9
-9
vosk.d.ts
src/renderer/plugins/asr/vosk/vosk.d.ts
+1
-1
vosk.js
src/renderer/plugins/asr/vosk/vosk.js
+600
-508
worker.d.ts
src/renderer/plugins/asr/vosk/worker.d.ts
+23
-23
FetchTTS.ts
src/renderer/plugins/tts/FetchTTS.ts
+34
-22
index.ts
src/renderer/plugins/tts/index.ts
+1
-1
recognizer-processor.js
src/renderer/public/vosk/recognizer-processor.js
+33
-33
PhotoScreen.vue
src/renderer/screens/PhotoScreen.vue
+55
-35
ShowPhoto.vue
src/renderer/screens/ShowPhoto.vue
+172
-131
index.ts
src/renderer/store/index.ts
+3
-3
photo.ts
src/renderer/store/photo.ts
+14
-15
settings.ts
src/renderer/store/settings.ts
+44
-37
tsconfig.json
tsconfig.json
+1
-7
No files found.
.vscode/settings.json
View file @
fd1558a6
...
...
@@ -21,5 +21,5 @@
"cSpell.words"
:
[
"Vosk"
],
"editor.inlineSuggest.showToolbar"
:
"
always
"
"editor.inlineSuggest.showToolbar"
:
"
onHover
"
}
README.md
View file @
fd1558a6
# chartIP-Electron
\ No newline at end of file
# chartIP-Electron
buildAssets/builder/config.js
View file @
fd1558a6
...
...
@@ -69,7 +69,7 @@ const baseConfig = {
oneClick
:
true
},
linux
:
{
executableName
:
'
vutron
'
,
executableName
:
'
chartIP
'
,
icon
:
'buildAssets/icons'
,
category
:
'Utility'
,
target
:
[
...
...
src/main/IPCs.ts
View file @
fd1558a6
...
...
@@ -5,8 +5,7 @@ import Constants from './utils/Constants'
* IPC Communications
* */
export
default
class
IPCs
{
static
browserWindows
:
Map
<
string
,
BrowserWindow
[]
>
=
new
Map
();
static
browserWindows
:
Map
<
string
,
BrowserWindow
[]
>
=
new
Map
()
static
initialize
(
window
:
BrowserWindow
):
void
{
// Get application version
...
...
@@ -20,41 +19,57 @@ export default class IPCs {
})
// open new window
ipcMain
.
on
(
'openWindow'
,
async
(
event
,
url
:
string
,
options
:
BrowserWindowConstructorOptions
&
{
isCloseOther
:
boolean
})
=>
{
const
ops
=
Object
.
assign
({},
{
isCloseOther
:
true
,
frame
:
false
,
useContentSize
:
true
,
webPreferences
:
Constants
.
DEFAULT_WEB_PREFERENCES
},
options
);
ipcMain
.
on
(
'openWindow'
,
async
(
event
,
url
:
string
,
options
:
BrowserWindowConstructorOptions
&
{
isCloseOther
:
boolean
}
)
=>
{
const
ops
=
Object
.
assign
(
{},
{
isCloseOther
:
true
,
frame
:
false
,
useContentSize
:
true
,
webPreferences
:
Constants
.
DEFAULT_WEB_PREFERENCES
},
options
)
if
(
IPCs
.
browserWindows
.
has
(
url
)
&&
ops
.
isCloseOther
)
{
const
wins
=
IPCs
.
browserWindows
.
get
(
url
);
wins
?.
forEach
(
w
=>
!
w
.
isDestroyed
()
&&
w
.
close
());
IPCs
.
browserWindows
.
set
(
url
,
[]);
}
if
(
IPCs
.
browserWindows
.
has
(
url
)
&&
ops
.
isCloseOther
)
{
const
wins
=
IPCs
.
browserWindows
.
get
(
url
)
wins
?.
forEach
((
w
)
=>
!
w
.
isDestroyed
()
&&
w
.
close
())
IPCs
.
browserWindows
.
set
(
url
,
[])
}
const
win
=
new
BrowserWindow
(
ops
)
const
win
=
new
BrowserWindow
(
ops
)
win
.
setMenu
(
null
)
win
.
setMenu
(
null
)
win
.
once
(
'ready-to-show'
,
():
void
=>
{
win
.
setAlwaysOnTop
(
true
)
win
.
show
()
win
.
focus
()
win
.
setAlwaysOnTop
(
false
)
})
win
.
once
(
'ready-to-show'
,
():
void
=>
{
win
.
setAlwaysOnTop
(
true
)
win
.
show
()
win
.
focus
()
win
.
setAlwaysOnTop
(
false
)
})
win
.
webContents
.
on
(
'did-frame-finish-load'
,
():
void
=>
{
if
(
Constants
.
IS_DEV_ENV
)
{
win
.
webContents
.
openDevTools
()
}
})
win
.
webContents
.
on
(
'did-frame-finish-load'
,
():
void
=>
{
if
(
Constants
.
IS_DEV_ENV
)
{
win
.
webContents
.
openDevTools
()
}
})
await
win
.
loadURL
(
`
${
Constants
.
APP_INDEX_URL_DEV
}${
url
}
`
)
await
win
.
loadURL
(
`
${
Constants
.
APP_INDEX_URL_DEV
}${
url
}
`
)
if
(
!
IPCs
.
browserWindows
.
has
(
url
))
{
IPCs
.
browserWindows
.
set
(
url
,
[]);
}
if
(
!
IPCs
.
browserWindows
.
has
(
url
))
{
IPCs
.
browserWindows
.
set
(
url
,
[])
}
IPCs
.
browserWindows
.
get
(
url
)?.
push
(
win
);
IPCs
.
browserWindows
.
get
(
url
)?.
push
(
win
)
return
win
;
})
return
win
}
)
}
}
src/renderer/components/layout/DefaultLayout.vue
View file @
fd1558a6
<
script
setup
lang=
"ts"
>
import
HeaderLayout
from
'@/renderer/components/layout/HeaderLayout.vue'
import
{
ref
}
from
'vue'
;
import
{
ref
}
from
'vue'
import
{
useRouter
}
from
'vue-router'
const
router
=
useRouter
()
const
isHeader
=
ref
(
true
)
;
const
isHeader
=
ref
(
true
)
router
.
beforeEach
((
guard
)
=>
{
isHeader
.
value
=
typeof
guard
.
meta
.
isHeader
===
'boolean'
?
(
guard
.
meta
.
isHeader
as
boolean
)
:
true
;
isHeader
.
value
=
typeof
guard
.
meta
.
isHeader
===
'boolean'
?
(
guard
.
meta
.
isHeader
as
boolean
)
:
true
})
</
script
>
<
template
>
...
...
src/renderer/components/layout/HeaderLayout.vue
View file @
fd1558a6
<
script
setup
lang=
"tsx"
>
import
{
computed
,
ref
}
from
'vue'
;
import
{
computed
,
ref
}
from
'vue'
import
{
useRoute
,
useRouter
}
from
'vue-router'
import
useStore
from
'@/renderer/store'
;
import
{
storeToRefs
}
from
'pinia'
;
import
useStore
from
'@/renderer/store'
import
{
storeToRefs
}
from
'pinia'
import
{
audioAiTTS
}
from
'@/renderer/plugins/tts'
const
router
=
useRouter
()
const
route
:
any
=
useRoute
()
const
{
settings
}
=
useStore
()
;
const
setting
=
storeToRefs
(
settings
)
;
const
{
settings
}
=
useStore
()
const
setting
=
storeToRefs
(
settings
)
settings
.
getSource
()
;
settings
.
getSource
()
const
handleRoute
=
(
path
:
string
):
void
=>
{
router
.
push
(
path
)
...
...
@@ -25,8 +25,8 @@ const asrItems = ref([
'vosk_asr'
,
'xf_asr'
// 'Whisper Api'
])
;
const
asrSelect
=
ref
(
setting
.
asr
)
;
])
const
asrSelect
=
ref
(
setting
.
asr
)
const
source
=
computed
(()
=>
{
return
setting
.
source
.
value
.
map
(({
sourceId
,
sourceName
,
description
,
sex
})
=>
{
...
...
@@ -35,17 +35,22 @@ const source = computed(() => {
value
:
sourceId
,
title
:
`
${
sourceName
}
-
${
_sex
}
-
${
description
}
`
}
})
;
})
;
})
})
async
function
changeSource
()
{
const
tone
=
setting
.
source
.
value
.
find
(({
sourceId
})
=>
setting
.
selectSource
.
value
===
sourceId
);
if
(
!
tone
)
return
;
const
res
=
await
audioAiTTS
({
host
:
settings
.
ttsHost
,
text
:
'你好,今天天气怎么样?'
,
speed
:
5.5
,
speaker
:
tone
.
sourceId
,
provider
:
tone
.
provider
});
console
.
log
(
res
);
const
tone
=
setting
.
source
.
value
.
find
(({
sourceId
})
=>
setting
.
selectSource
.
value
===
sourceId
)
if
(
!
tone
)
return
const
res
=
await
audioAiTTS
({
host
:
settings
.
ttsHost
,
text
:
'你好,今天天气怎么样?'
,
speed
:
5.5
,
speaker
:
tone
.
sourceId
,
provider
:
tone
.
provider
})
console
.
log
(
res
)
}
</
script
>
<
template
>
<v-app-bar
color=
"#d71b1b"
density=
"compact"
class=
"header"
>
...
...
@@ -69,29 +74,20 @@ async function changeSource() {
<v-dialog
width=
"600"
>
<template
#
activator=
"
{ props }">
<v-btn
v-bind=
"props"
color=
"#fff"
class=
"settings"
>
<v-icon
start
icon=
"mdi-wrench"
></v-icon>
<v-btn
v-bind=
"props"
color=
"#fff"
class=
"settings"
>
<v-icon
start
icon=
"mdi-wrench"
></v-icon>
配置
</v-btn>
</
template
>
<
template
#
default=
"{ isActive }"
>
<v-card
title=
"配置"
>
<v-sheet
width=
"500"
class=
"mx-auto mt-6"
>
<v-form
ref=
"form"
>
<v-select
v-model=
"setting.asr.value"
:items=
"asrItems"
:rules=
"[
v
=> !!v || '请选择 Asr']"
:rules=
"[
(v)
=> !!v || '请选择 Asr']"
label=
"语音识别(ASR)"
required
></v-select>
...
...
@@ -107,9 +103,7 @@ async function changeSource() {
<v-text-field
label=
"TTS 域名"
:rules=
"[
value => !!value || 'TTS 域名必填',
]"
:rules=
"[(value) => !!value || 'TTS 域名必填']"
hide-details=
"auto"
:model-value=
"setting.ttsHost"
></v-text-field>
...
...
@@ -118,26 +112,21 @@ async function changeSource() {
v-model=
"setting.selectSource.value"
class=
"mt-6"
:items=
"source"
:rules=
"[
v
=> !!v || '请选择音色']"
:rules=
"[
(v)
=> !!v || '请选择音色']"
label=
"TTS 音色"
required
@
update:model-value=
"changeSource"
></v-select>
</v-form>
</v-sheet>
<v-card-actions>
<v-spacer></v-spacer>
<v-btn
text=
"关闭"
@
click=
"isActive.value = false"
></v-btn>
<v-btn
text=
"关闭"
@
click=
"isActive.value = false"
></v-btn>
</v-card-actions>
</v-card>
</template>
</v-dialog>
</template>
</v-app-bar>
</template>
...
...
src/renderer/main.ts
View file @
fd1558a6
...
...
@@ -7,8 +7,8 @@ import vuetify from '@/renderer/plugins/vuetify'
import
i18n
from
'@/renderer/plugins/i18n'
import
piniaPluginPersistedstate
from
'pinia-plugin-persistedstate'
const
pinia
=
createPinia
()
;
pinia
.
use
(
piniaPluginPersistedstate
)
;
const
pinia
=
createPinia
()
pinia
.
use
(
piniaPluginPersistedstate
)
// Add API key defined in contextBridge to window object type
declare
global
{
...
...
src/renderer/plugins/asr/index.ts
View file @
fd1558a6
export
*
as
Vosk
from
'./vosk/vosk'
export
type
*
from
'./vosk/vosk'
\ No newline at end of file
export
type
*
from
'./vosk/vosk'
src/renderer/plugins/asr/vosk/interfaces.d.ts
View file @
fd1558a6
export
interface
ClientMessageLoad
{
action
:
"load"
;
modelUrl
:
string
;
action
:
'load'
modelUrl
:
string
}
export
interface
ClientMessageTerminate
{
action
:
"terminate"
;
action
:
'terminate'
}
export
interface
ClientMessageRecognizerSet
{
action
:
"set"
;
recognizerId
:
string
;
key
:
"words"
;
value
:
boolean
;
action
:
'set'
recognizerId
:
string
key
:
'words'
value
:
boolean
}
export
interface
ClientMessageGenericSet
{
action
:
"set"
;
key
:
"logLevel"
;
value
:
number
;
action
:
'set'
key
:
'logLevel'
value
:
number
}
export
declare
type
ClientMessageSet
=
ClientMessageRecognizerSet
|
ClientMessageGenericSet
;
export
declare
type
ClientMessageSet
=
ClientMessageRecognizerSet
|
ClientMessageGenericSet
export
interface
ClientMessageAudioChunk
{
action
:
"audioChunk"
;
recognizerId
:
string
;
data
:
Float32Array
;
sampleRate
:
number
;
action
:
'audioChunk'
recognizerId
:
string
data
:
Float32Array
sampleRate
:
number
}
export
interface
ClientMessageCreateRecognizer
{
action
:
"create"
;
recognizerId
:
string
;
sampleRate
:
number
;
grammar
?:
string
;
action
:
'create'
recognizerId
:
string
sampleRate
:
number
grammar
?:
string
}
export
interface
ClientMessageRetrieveFinalResult
{
action
:
"retrieveFinalResult"
;
recognizerId
:
string
;
action
:
'retrieveFinalResult'
recognizerId
:
string
}
export
interface
ClientMessageRemoveRecognizer
{
action
:
"remove"
;
recognizerId
:
string
;
action
:
'remove'
recognizerId
:
string
}
export
declare
type
ClientMessage
=
ClientMessageTerminate
|
ClientMessageLoad
|
ClientMessageCreateRecognizer
|
ClientMessageAudioChunk
|
ClientMessageSet
|
ClientMessageRetrieveFinalResult
|
ClientMessageRemoveRecognizer
;
export
declare
type
ClientMessage
=
|
ClientMessageTerminate
|
ClientMessageLoad
|
ClientMessageCreateRecognizer
|
ClientMessageAudioChunk
|
ClientMessageSet
|
ClientMessageRetrieveFinalResult
|
ClientMessageRemoveRecognizer
export
declare
namespace
ClientMessage
{
function
isTerminateMessage
(
message
:
ClientMessage
):
message
is
ClientMessageTerminate
;
function
isLoadMessage
(
message
:
ClientMessage
):
message
is
ClientMessageLoad
;
function
isSetMessage
(
message
:
ClientMessage
):
message
is
ClientMessageSet
;
function
isAudioChunkMessage
(
message
:
ClientMessage
):
message
is
ClientMessageAudioChunk
;
function
isRecognizerCreateMessage
(
message
:
ClientMessage
):
message
is
ClientMessageCreateRecognizer
;
function
isRecognizerRetrieveFinalResultMessage
(
message
:
ClientMessage
):
message
is
ClientMessageRetrieveFinalResult
;
function
isRecognizerRemoveMessage
(
message
:
ClientMessage
):
message
is
ClientMessageRemoveRecognizer
;
function
isTerminateMessage
(
message
:
ClientMessage
):
message
is
ClientMessageTerminate
function
isLoadMessage
(
message
:
ClientMessage
):
message
is
ClientMessageLoad
function
isSetMessage
(
message
:
ClientMessage
):
message
is
ClientMessageSet
function
isAudioChunkMessage
(
message
:
ClientMessage
):
message
is
ClientMessageAudioChunk
function
isRecognizerCreateMessage
(
message
:
ClientMessage
):
message
is
ClientMessageCreateRecognizer
function
isRecognizerRetrieveFinalResultMessage
(
message
:
ClientMessage
):
message
is
ClientMessageRetrieveFinalResult
function
isRecognizerRemoveMessage
(
message
:
ClientMessage
):
message
is
ClientMessageRemoveRecognizer
}
export
interface
ServerMessageLoadResult
{
event
:
"load"
;
result
:
boolean
;
event
:
'load'
result
:
boolean
}
export
interface
ServerMessageError
{
event
:
"error"
;
recognizerId
?:
string
;
error
:
string
;
event
:
'error'
recognizerId
?:
string
error
:
string
}
export
interface
ServerMessageResult
{
event
:
"result"
;
recognizerId
:
string
;
result
:
{
result
:
Array
<
{
conf
:
number
;
start
:
number
;
end
:
number
;
word
:
string
;
}
>
;
text
:
string
;
};
event
:
'result'
recognizerId
:
string
result
:
{
result
:
Array
<
{
conf
:
number
start
:
number
end
:
number
word
:
string
}
>
text
:
string
}
}
export
interface
ServerMessagePartialResult
{
event
:
"partialresult"
;
recognizerId
:
string
;
result
:
{
partial
:
string
;
};
event
:
'partialresult'
recognizerId
:
string
result
:
{
partial
:
string
}
}
export
declare
type
ModelMessage
=
ServerMessageLoadResult
|
ServerMessageError
;
export
declare
type
ModelMessage
=
ServerMessageLoadResult
|
ServerMessageError
export
declare
namespace
ModelMessage
{
function
isLoadResult
(
message
:
any
):
message
is
ServerMessageLoadResult
;
function
isLoadResult
(
message
:
any
):
message
is
ServerMessageLoadResult
}
export
declare
type
RecognizerMessage
=
ServerMessagePartialResult
|
ServerMessageResult
|
ServerMessageError
;
export
declare
type
RecognizerEvent
=
RecognizerMessage
[
"event"
];
export
declare
type
ServerMessage
=
ModelMessage
|
RecognizerMessage
;
export
declare
type
RecognizerMessage
=
|
ServerMessagePartialResult
|
ServerMessageResult
|
ServerMessageError
export
declare
type
RecognizerEvent
=
RecognizerMessage
[
'event'
]
export
declare
type
ServerMessage
=
ModelMessage
|
RecognizerMessage
export
declare
namespace
ServerMessage
{
function
isRecognizerMessage
(
message
:
ServerMessage
):
message
is
RecognizerMessage
;
function
isResult
(
message
:
any
):
message
is
ServerMessageResult
;
function
isPartialResult
(
message
:
any
):
message
is
ServerMessagePartialResult
;
function
isRecognizerMessage
(
message
:
ServerMessage
):
message
is
RecognizerMessage
function
isResult
(
message
:
any
):
message
is
ServerMessageResult
function
isPartialResult
(
message
:
any
):
message
is
ServerMessagePartialResult
}
src/renderer/plugins/asr/vosk/model.d.ts
View file @
fd1558a6
import
{
ModelMessage
,
RecognizerEvent
,
RecognizerMessage
}
from
"./interfaces"
;
export
*
from
"./interfaces"
;
import
{
ModelMessage
,
RecognizerEvent
,
RecognizerMessage
}
from
'./interfaces'
export
*
from
'./interfaces'
export
declare
class
Model
extends
EventTarget
{
private
modelUrl
;
private
worker
;
private
_ready
;
private
messagePort
;
private
logger
;
private
recognizers
;
constructor
(
modelUrl
:
string
,
logLevel
?:
number
);
private
initialize
;
private
postMessage
;
private
handleMessage
;
on
(
event
:
ModelMessage
[
"event"
],
listener
:
(
message
:
ModelMessage
)
=>
void
):
void
;
registerPort
(
port
:
MessagePort
):
void
;
private
forwardMessage
;
get
ready
():
boolean
;
terminate
():
void
;
setLogLevel
(
level
:
number
):
void
;
registerRecognizer
(
recognizer
:
KaldiRecognizer
):
void
;
unregisterRecognizer
(
recognizerId
:
string
):
void
;
/**
* KaldiRecognizer anonymous class
*/
get
KaldiRecognizer
():
{
new
(
sampleRate
:
number
,
grammar
?:
string
):
{
id
:
string
;
on
(
event
:
RecognizerEvent
,
listener
:
(
message
:
RecognizerMessage
)
=>
void
):
void
;
setWords
(
words
:
boolean
):
void
;
acceptWaveform
(
buffer
:
AudioBuffer
):
void
;
acceptWaveformFloat
(
buffer
:
Float32Array
,
sampleRate
:
number
):
void
;
retrieveFinalResult
():
void
;
remove
():
void
;
addEventListener
(
type
:
string
,
callback
:
EventListenerOrEventListenerObject
|
null
,
options
?:
boolean
|
AddEventListenerOptions
|
undefined
):
void
;
dispatchEvent
(
event
:
Event
):
boolean
;
removeEventListener
(
type
:
string
,
callback
:
EventListenerOrEventListenerObject
|
null
,
options
?:
boolean
|
EventListenerOptions
|
undefined
):
void
;
};
};
private
modelUrl
private
worker
private
_ready
private
messagePort
private
logger
private
recognizers
constructor
(
modelUrl
:
string
,
logLevel
?:
number
)
private
initialize
private
postMessage
private
handleMessage
on
(
event
:
ModelMessage
[
'event'
],
listener
:
(
message
:
ModelMessage
)
=>
void
):
void
registerPort
(
port
:
MessagePort
):
void
private
forwardMessage
get
ready
():
boolean
terminate
():
void
setLogLevel
(
level
:
number
):
void
registerRecognizer
(
recognizer
:
KaldiRecognizer
):
void
unregisterRecognizer
(
recognizerId
:
string
):
void
/**
* KaldiRecognizer anonymous class
*/
get
KaldiRecognizer
():
{
new
(
sampleRate
:
number
,
grammar
?:
string
):
{
id
:
string
on
(
event
:
RecognizerEvent
,
listener
:
(
message
:
RecognizerMessage
)
=>
void
):
void
setWords
(
words
:
boolean
):
void
acceptWaveform
(
buffer
:
AudioBuffer
):
void
acceptWaveformFloat
(
buffer
:
Float32Array
,
sampleRate
:
number
):
void
retrieveFinalResult
():
void
remove
():
void
addEventListener
(
type
:
string
,
callback
:
EventListenerOrEventListenerObject
|
null
,
options
?:
boolean
|
AddEventListenerOptions
|
undefined
):
void
dispatchEvent
(
event
:
Event
):
boolean
removeEventListener
(
type
:
string
,
callback
:
EventListenerOrEventListenerObject
|
null
,
options
?:
boolean
|
EventListenerOptions
|
undefined
):
void
}
}
}
export
declare
type
KaldiRecognizer
=
InstanceType
<
Model
[
"KaldiRecognizer"
]
>
;
export
declare
function
createModel
(
modelUrl
:
string
,
logLevel
?:
number
):
Promise
<
Model
>
;
export
declare
type
KaldiRecognizer
=
InstanceType
<
Model
[
'KaldiRecognizer'
]
>
export
declare
function
createModel
(
modelUrl
:
string
,
logLevel
?:
number
):
Promise
<
Model
>
src/renderer/plugins/asr/vosk/utils/logging.d.ts
View file @
fd1558a6
export
declare
class
Logger
{
private
logLevel
;
constructor
(
logLevel
?:
number
);
getLogLevel
():
number
;
setLogLevel
(
level
:
number
):
void
;
error
(
message
:
string
):
void
;
warn
(
message
:
string
):
void
;
info
(
message
:
string
):
void
;
verbose
(
message
:
string
):
void
;
debug
(
message
:
string
):
void
;
private
logLevel
constructor
(
logLevel
?:
number
)
getLogLevel
():
number
setLogLevel
(
level
:
number
):
void
error
(
message
:
string
):
void
warn
(
message
:
string
):
void
info
(
message
:
string
):
void
verbose
(
message
:
string
):
void
debug
(
message
:
string
):
void
}
src/renderer/plugins/asr/vosk/vosk.d.ts
View file @
fd1558a6
export
*
from
"./model"
;
export
*
from
'./model'
src/renderer/plugins/asr/vosk/vosk.js
View file @
fd1558a6
This diff is collapsed.
Click to expand it.
src/renderer/plugins/asr/vosk/worker.d.ts
View file @
fd1558a6
import
*
as
VoskWasm
from
"./vosk-wasm"
;
import
*
as
VoskWasm
from
'./vosk-wasm'
export
interface
Recognizer
{
id
:
string
;
buffAddr
?:
number
;
buffSize
?:
number
;
recognizer
:
VoskWasm
.
Recognizer
;
sampleRate
:
number
;
words
?:
boolean
;
grammar
?:
string
;
id
:
string
buffAddr
?:
number
buffSize
?:
number
recognizer
:
VoskWasm
.
Recognizer
sampleRate
:
number
words
?:
boolean
grammar
?:
string
}
export
declare
class
RecognizerWorker
{
private
Vosk
;
private
model
;
private
recognizers
;
private
logger
;
constructor
();
private
handleMessage
;
private
load
;
private
allocateBuffer
;
private
freeBuffer
;
private
createRecognizer
;
private
setConfiguration
;
private
processAudioChunk
;
private
retrieveFinalResult
;
private
removeRecognizer
;
private
terminate
;
private
Vosk
private
model
private
recognizers
private
logger
constructor
()
private
handleMessage
private
load
private
allocateBuffer
private
freeBuffer
private
createRecognizer
private
setConfiguration
private
processAudioChunk
private
retrieveFinalResult
private
removeRecognizer
private
terminate
}
src/renderer/plugins/tts/FetchTTS.ts
View file @
fd1558a6
export
async
function
audioAiTTS
({
host
,
text
,
speaker
,
speed
,
provider
}:
{
host
:
string
;
text
:
string
;
speaker
:
string
;
speed
:
number
;
provider
:
number
;
})
{
const
resp
=
await
fetch
(
`
${
host
}
/api/live/audioAI`
,
{
"headers"
:
{
"accept"
:
"application/json, text/plain, */*"
,
"content-type"
:
"application/json"
,
},
body
:
JSON
.
stringify
({
type
:
1
,
text
,
speaker
,
speed
,
provider
,
pause_points
:
[],
sceneType
:
1
,
gender
:
1
export
async
function
audioAiTTS
({
host
,
text
,
speaker
,
speed
,
provider
}:
{
host
:
string
text
:
string
speaker
:
string
speed
:
number
provider
:
number
})
{
const
resp
=
await
fetch
(
`
${
host
}
/api/live/audioAI`
,
{
headers
:
{
accept
:
'application/json, text/plain, */*'
,
'content-type'
:
'application/json'
},
body
:
JSON
.
stringify
({
type
:
1
,
text
,
speaker
,
speed
,
provider
,
pause_points
:
[],
sceneType
:
1
,
gender
:
1
}),
method
:
"POST"
,
mode
:
"cors"
,
})
;
method
:
'POST'
,
mode
:
'cors'
})
const
res
=
await
resp
.
json
();
if
(
res
.
code
!==
200
)
throw
new
Error
(
JSON
.
stringify
(
res
));
return
res
.
data
;
}
\ No newline at end of file
const
res
=
await
resp
.
json
()
if
(
res
.
code
!==
200
)
throw
new
Error
(
JSON
.
stringify
(
res
))
return
res
.
data
}
src/renderer/plugins/tts/index.ts
View file @
fd1558a6
export
*
from
'./FetchTTS'
\ No newline at end of file
export
*
from
'./FetchTTS'
src/renderer/public/vosk/recognizer-processor.js
View file @
fd1558a6
class
RecognizerAudioProcessor
extends
AudioWorkletProcessor
{
constructor
(
options
)
{
super
(
options
);
this
.
port
.
onmessage
=
this
.
_processMessage
.
bind
(
this
);
}
_processMessage
(
event
)
{
// console.debug(`Received event ${JSON.stringify(event.data, null, 2)}`);
if
(
event
.
data
.
action
===
"init"
)
{
this
.
_recognizerId
=
event
.
data
.
recognizerId
;
this
.
_recognizerPort
=
event
.
ports
[
0
];
}
constructor
(
options
)
{
super
(
options
)
this
.
port
.
onmessage
=
this
.
_processMessage
.
bind
(
this
)
}
_processMessage
(
event
)
{
// console.debug(`Received event ${JSON.stringify(event.data, null, 2)}`);
if
(
event
.
data
.
action
===
'init'
)
{
this
.
_recognizerId
=
event
.
data
.
recognizerId
this
.
_recognizerPort
=
event
.
ports
[
0
]
}
process
(
inputs
,
outputs
,
parameters
)
{
const
data
=
inputs
[
0
][
0
];
if
(
this
.
_recognizerPort
&&
data
)
{
// AudioBuffer samples are represented as floating point numbers between -1.0 and 1.0 whilst
// Kaldi expects them to be between -32768 and 32767 (the range of a signed int16)
const
audioArray
=
data
.
map
((
value
)
=>
value
*
0x8000
);
this
.
_recognizerPort
.
postMessage
(
{
action
:
"audioChunk"
,
data
:
audioArray
,
recognizerId
:
this
.
_recognizerId
,
sampleRate
,
// Part of AudioWorkletGlobalScope
},
{
transfer
:
[
audioArray
.
buffer
],
}
);
}
process
(
inputs
,
outputs
,
parameters
)
{
const
data
=
inputs
[
0
][
0
]
if
(
this
.
_recognizerPort
&&
data
)
{
// AudioBuffer samples are represented as floating point numbers between -1.0 and 1.0 whilst
// Kaldi expects them to be between -32768 and 32767 (the range of a signed int16)
const
audioArray
=
data
.
map
((
value
)
=>
value
*
0x8000
)
this
.
_recognizerPort
.
postMessage
(
{
action
:
'audioChunk'
,
data
:
audioArray
,
recognizerId
:
this
.
_recognizerId
,
sampleRate
// Part of AudioWorkletGlobalScope
},
{
transfer
:
[
audioArray
.
buffer
]
}
return
true
;
)
}
return
true
}
}
registerProcessor
(
'recognizer-processor'
,
RecognizerAudioProcessor
)
\ No newline at end of file
registerProcessor
(
'recognizer-processor'
,
RecognizerAudioProcessor
)
src/renderer/screens/PhotoScreen.vue
View file @
fd1558a6
...
...
@@ -5,11 +5,11 @@
// import { useCounterStore } from '@/renderer/store/counter'
// import { storeToRefs } from 'pinia'
import
{
onMounted
,
ref
}
from
'vue'
import
useStore
from
'@/renderer/store'
;
import
{
storeToRefs
}
from
'pinia'
;
import
useStore
from
'@/renderer/store'
import
{
storeToRefs
}
from
'pinia'
const
{
photo
:
usePhoto
}
=
useStore
()
;
const
photo
=
storeToRefs
(
usePhoto
)
;
const
{
photo
:
usePhoto
}
=
useStore
()
const
photo
=
storeToRefs
(
usePhoto
)
// const { availableLocales } = useI18n()
// const { counterIncrease } = useCounterStore()
...
...
@@ -20,70 +20,90 @@ const photo = storeToRefs(usePhoto);
onMounted
(():
void
=>
{
// languages.value = availableLocales
// window.mainApi.receive('msgReceivedVersion', (event: Event, version: string) => {
// appVersion.value = version
// })
// window.mainApi.send('msgRequestGetVersion')
})
async
function
handleOpen
(
event
:
Event
,
url
:
string
)
{
await
window
.
mainApi
.
send
(
'openWindow'
,
`#show?url=
${
url
}
`
,
{
width
:
window
.
screen
.
width
/
4
,
height
:
window
.
screen
.
height
});
async
function
handleOpen
(
event
:
Event
,
url
:
string
)
{
await
window
.
mainApi
.
send
(
'openWindow'
,
`#show?url=
${
url
}
`
,
{
width
:
window
.
screen
.
width
/
4
,
height
:
window
.
screen
.
height
})
}
const
validateURL
=
(
url
:
string
)
=>
{
const
regex
=
/^
(
https
?
|ftp
)
:
\/\/([\w/\-
?=%.
]
+
\.[\w/\-
?=%.
]
+
)
$/
;
return
regex
.
test
(
url
)
;
const
regex
=
/^
(
https
?
|ftp
)
:
\/\/([\w/\-
?=%.
]
+
\.[\w/\-
?=%.
]
+
)
$/
return
regex
.
test
(
url
)
}
const
urlValue
=
ref
(
''
)
;
const
imgLoading
=
ref
(
false
)
;
const
urlValue
=
ref
(
''
)
const
imgLoading
=
ref
(
false
)
async
function
appendPhoto
(
url
:
string
)
{
urlValue
.
value
=
url
;
urlValue
.
value
=
url
if
(
!
validateURL
(
url
))
return
'请输入正确的 url!如(url(https://xxx.png)'
try
{
imgLoading
.
value
=
true
;
const
img
=
new
Image
()
;
img
.
src
=
url
;
imgLoading
.
value
=
true
const
img
=
new
Image
()
img
.
src
=
url
await
new
Promise
((
resolve
,
reject
)
=>
{
img
.
onload
=
resolve
;
img
.
onerror
=
reject
;
})
;
imgLoading
.
value
=
false
;
img
.
onload
=
resolve
img
.
onerror
=
reject
})
imgLoading
.
value
=
false
}
catch
(
error
)
{
imgLoading
.
value
=
false
;
imgLoading
.
value
=
false
return
'图片加载失败!'
}
photo
.
list
.
value
.
push
({
url
})
;
photo
.
list
.
value
.
push
({
url
})
urlValue
.
value
=
''
return
true
;
return
true
}
function
removePhoto
(
index
:
number
)
{
photo
.
list
.
value
.
splice
(
index
,
1
)
}
</
script
>
<
template
>
<v-container
class=
"d-flex mt-6 pb-0"
>
<v-text-field
label=
"自定义照片 url(https://xxx.png)"
:model-value=
"urlValue"
:loading=
"imgLoading"
:rules=
"[ v => appendPhoto(v) ]"
validate-on=
"blur lazy"
></v-text-field>
<v-text-field
label=
"自定义照片 url(https://xxx.png)"
:model-value=
"urlValue"
:loading=
"imgLoading"
:rules=
"[(v) => appendPhoto(v)]"
validate-on=
"blur lazy"
></v-text-field>
</v-container>
<v-container
class=
"d-flex flex-wrap"
>
<v-sheet
v-for=
"(item, index) in photo.list.value"
:key=
"item.url"
v-ripple
:elevation=
"3"
width=
"200"
class=
"d-flex spacing-playground pa-6 mr-4 mt-4"
rounded
>
<v-img
:width=
"200"
aspect-ratio=
"1/1"
cover
:src=
"item.url"
@
click=
"handleOpen($event, item.url)"
></v-img>
<v-btn
density=
"compact"
elevation=
"1"
icon=
"mdi-close"
class=
"mt-n7"
@
click=
"removePhoto(index)"
></v-btn>
</v-sheet>
<v-sheet
v-for=
"(item, index) in photo.list.value"
:key=
"item.url"
v-ripple
:elevation=
"3"
width=
"200"
class=
"d-flex spacing-playground pa-6 mr-4 mt-4"
rounded
>
<v-img
:width=
"200"
aspect-ratio=
"1/1"
cover
:src=
"item.url"
@
click=
"handleOpen($event, item.url)"
></v-img>
<v-btn
density=
"compact"
elevation=
"1"
icon=
"mdi-close"
class=
"mt-n7"
@
click=
"removePhoto(index)"
></v-btn>
</v-sheet>
</v-container>
</
template
>
src/renderer/screens/ShowPhoto.vue
View file @
fd1558a6
This diff is collapsed.
Click to expand it.
src/renderer/store/index.ts
View file @
fd1558a6
import
useSettings
from
'./settings'
;
import
usePhoto
from
'./photo'
;
import
useSettings
from
'./settings'
import
usePhoto
from
'./photo'
export
default
function
useStore
()
{
return
{
settings
:
useSettings
(),
photo
:
usePhoto
()
}
}
\ No newline at end of file
}
src/renderer/store/photo.ts
View file @
fd1558a6
...
...
@@ -6,20 +6,19 @@ type IPhoto = {
const
usePhotoStore
=
defineStore
(
'photo'
,
{
persist
:
true
,
state
:
()
=>
({
list
:
[
{
url
:
'https://resources.laihua.com/2023-11-2/93ffb6a7-ae93-4918-944e-877016ba266b.png'
},
{
url
:
'https://resources.laihua.com/2023-6-19/6fa9a127-2ce5-43ea-a543-475bf9354eda.png'
}
]
}
as
IPhoto
),
getters
:
{
},
actions
:
{
}
state
:
()
=>
({
list
:
[
{
url
:
'https://resources.laihua.com/2023-11-2/93ffb6a7-ae93-4918-944e-877016ba266b.png'
},
{
url
:
'https://resources.laihua.com/2023-6-19/6fa9a127-2ce5-43ea-a543-475bf9354eda.png'
}
]
})
as
IPhoto
,
getters
:
{},
actions
:
{}
})
export
default
usePhotoStore
\ No newline at end of file
export
default
usePhotoStore
src/renderer/store/settings.ts
View file @
fd1558a6
import
{
defineStore
}
from
'pinia'
export
type
ISettings
=
{
asr
:
'vosk_asr'
|
'xf_asr'
;
voskModels
:
string
[];
voskSelectModel
:
string
;
ttsHost
:
string
;
source
:
{
sourceName
:
string
;
sourceId
:
string
;
provider
:
number
;
speaker
:
string
;
description
:
string
;
sex
:
1
|
0
}[];
selectSource
:
string
;
asr
:
'vosk_asr'
|
'xf_asr'
voskModels
:
string
[]
voskSelectModel
:
string
ttsHost
:
string
source
:
{
sourceName
:
string
sourceId
:
string
provider
:
number
speaker
:
string
description
:
string
sex
:
1
|
0
}[]
selectSource
:
string
}
const
useSettingsStore
=
defineStore
(
'settings'
,
{
persist
:
true
,
state
:
()
=>
({
asr
:
'vosk_asr'
,
voskModels
:
[
'vosk-model-small-ca-0.4.tar.gz'
,
'vosk-model-small-cn-0.3
.tar.gz'
,
'vosk-model-small-de-0.15
.tar.gz'
,
'vosk-model-small-en-in-0.4
.tar.gz'
,
'vosk-model-small-en-us-0.15
.tar.gz'
,
'vosk-model-small-es-0.3
.tar.gz'
,
'vosk-model-small-fa-0.4
.tar.gz'
,
'vosk-model-small-fr-pguyot-0.3
.tar.gz'
,
'vosk-model-small-it-0.4
.tar.gz'
,
'vosk-model-small-pt-0.3
.tar.gz'
,
'vosk-model-small-ru-0.4
.tar.gz'
,
'vosk-model-small-tr-0.3
.tar.gz'
,
'vosk-model-small-vn-0.3.tar.gz'
],
voskSelectModel
:
'vosk-model-small-cn-0.3.tar.gz'
,
ttsHost
:
'https://beta.laihua.com
'
,
source
:
[]
,
selectSource
:
''
}
as
ISettings
),
getters
:
{
},
state
:
()
=>
({
asr
:
'vosk_asr'
,
voskModels
:
[
'vosk-model-small-ca-0.4
.tar.gz'
,
'vosk-model-small-cn-0.3
.tar.gz'
,
'vosk-model-small-de-0.15
.tar.gz'
,
'vosk-model-small-en-in-0.4
.tar.gz'
,
'vosk-model-small-en-us-0.15
.tar.gz'
,
'vosk-model-small-es-0.3
.tar.gz'
,
'vosk-model-small-fa-0.4
.tar.gz'
,
'vosk-model-small-fr-pguyot-0.3
.tar.gz'
,
'vosk-model-small-it-0.4
.tar.gz'
,
'vosk-model-small-pt-0.3
.tar.gz'
,
'vosk-model-small-ru-0.4
.tar.gz'
,
'vosk-model-small-tr-0.3.tar.gz'
,
'vosk-model-small-vn-0.3.tar.gz'
]
,
voskSelectModel
:
'vosk-model-small-cn-0.3.tar.gz
'
,
ttsHost
:
'https://beta.laihua.com'
,
source
:
[],
selectSource
:
''
})
as
ISettings
,
getters
:
{
},
actions
:
{
async
getSource
()
{
const
resp
=
await
fetch
(
`
${
this
.
$state
.
ttsHost
}
/api/live/audioAI/source?platform=31`
,
{
"method"
:
"GET"
,
"mode"
:
"cors"
,
})
;
method
:
'GET'
,
mode
:
'cors'
})
const
res
=
await
resp
.
json
()
;
const
res
=
await
resp
.
json
()
if
(
res
.
code
!==
200
)
return
;
this
.
source
=
res
.
data
;
if
(
res
.
code
!==
200
)
return
this
.
source
=
res
.
data
}
}
})
export
default
useSettingsStore
;
export
default
useSettingsStore
tsconfig.json
View file @
fd1558a6
...
...
@@ -26,11 +26,5 @@
"path"
:
"./tsconfig.node.json"
}
],
"exclude"
:
[
"node_modules"
,
"dist"
,
"rollup.config.js"
,
"*.json"
,
"*.js"
]
"exclude"
:
[
"node_modules"
,
"dist"
,
"rollup.config.js"
,
"*.json"
,
"*.js"
]
}
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment