Unofficial NovelAI General

Keep our community informed! This forum is for discussing and sharing vore-related information. Post any relevant material and/or links here, and engage in conversations!
Forum rules
This is for general discussion, if you found something you want to post, please use one of the upload forum, if you made something and want to share them, please use the work to be shared forum!

Unofficial NovelAI General

Postby AlluringPredation » Sun Oct 10, 2021 5:15 pm

Unofficial NovelAI General, where we can conspire together to cook NovelAI into something suitable for vore-related consumption.

NovelAI is an AI storytelling tool for anyone that desires to create and build their own stories, adventures or simply wants to fool around. By using our own trained AI models we offer an unprecedented level of freedom to the user. It has been described at different times as a writing assistant, a game, but, in truth, it's a limitless playground for your imagination. We hope to provide a user-friendly, customizable experience in line with your rights to privacy and transparency. We strive to become the leading AI storyteller on the market.
https://novelai.net/#/ - https://www.reddit.com/r/NovelAi/wiki/faq


Be aware that NovelAI is a paid service, with subscription plans for $10, $15, and $25. Mid tier should be more than satisfactory for many.

https://www.youtube.com/watch?v=Tekd5COUg8U&t=2643s Nice video of NovelAI in action as a writing tool assistant, exemplifies benefits and pitfalls you may come across. NovelAI doesn't do the work for you, quality of input affects the quality of output; Poor punctuation, grammar, and misspelling will be reflected by the AI's attempt to copy your style, and these issues will cause poor to unintelligible responses.

Screenshot below of me fooling around with NovelAI.
Spoiler: show
Know that Context sent to the AI is limited approximately 2500 words at any given time (actually 2048 Tokens), which is shared among Memory, Author's Note, LoreBook Entries, and the remaining bulk of Context space to recent Story text.

Image


About AI Language Models - Many newcomers are unaware the AI has no capacity for reasoning, it does know really know anything despite appearances otherwise. AI language models are insanely overcomplicated forms of auto-suggest on your phone's keyboard. It's using the available Context and a massive pool of data to sequentially randomly guess words likely to appear. It's the user's task to provide inspiration and lead the AI to lead to the desired result. It's helpful for details you may not have considered concerning character and setting, while throwing unexpected ideas to break through writer's block. Writing until you hit a stumbling point is an effective use of NovelAI.

=========================================================
Tools of NovelAI
Vore is woefully underrepresented in the training data. We're all aware vore itself goes against the natural process of consumption and breaking many rules of reality. Prepping NovelAI for vore thus presents some challenges. Yet the developers have equipped users tools for the task. Namely; Lorebooks, Phrase Biasing, and Modules. Each requiring an escalating amount of effort to properly utilize

Lorebook
- an absolutely neccessary for vore, and easiest to implement. Lacking proper understanding the AI needs a few examples of vore in Context, giving it instructions in how to proceed.
It is best to playtest repeated examples and keywords combinations that will yield desired results. Feel free to share examples and .Lorebook files to help one another out. But please make sure that the entries contained within a lorebook entry not contain copyrighted snippets of text, as lorebook can sometimes be utilized verbatim and this would constitute plagiarism.
Spoiler: show
Image


Phrase Bias - VERY new and technically challenging. It's benefits are being explored. It's basic fuction is adjusting odds of which certain words and phrases will occur in AI Output. Finding out what vorish terms to give a boost, and what to discourage is imperative towards improve the experience. It is VERY easy to ruin your experience with incorrect settings, luckily easy to fix (Adjust strength slider, or delete a bias list altogether.)

Examples
Spoiler: show

It is best to learn how tokens work before you attempt to Bias. Tokens are how the AI interprets text, with tokens comprising entire words or pieces of words. It is generally safe to Bias a word that appears as a whole. Though a word like suffocate will break into {suff|ocate}, which is generally safe
Positive Biasing, some words I'd like the encorage the AI's use of include: mouth, stomach, esophagus, melt, eat, etc.
Negative Biasing, words I like to limit the use and generally avoid: agony, nausea, blood, pain, hurts, chew, bite, crunch, oxygen, breath, gore, brain(s), child, kid

Related is Banned Tokens, where you can completely eliminate terms and phrases from ever occuring; "Won't kill you,you won't die, puke, vomit, regurgitate" I prefer what goes down to stay down, and not survive.


Modules, a sort of plug-in mini fine-tune, selected from default or custom imported from a drop-down menu. They are incredibly powerful in their influence, yet are the easiest tool for novice users. Except they take a lot of source material (dozens of stories) along with careful formatting to ensure they are prepared for training. A specialized vore module would make all the difference in assisting writers of all skill levels.

Modules are designed to pair certain words together, and enough pairs will form a sentence, onto paragraphs. Ideally this improves NovelAI's ability to follow along in specific situations, in this case vore. At some point I hope to have enough contributions to create a general-use-vore module, where no single writer nor themes (fantasy, sci-fi, horror, pokemon, etc.) has too much influence. Others may wish to have more specific themed modules, to a setting or special type of vore.

To this end I'm looking for multiple individuals to willingly contribute/link stories they would feel comfortable including within custom modules. The general agreement would be the story would only be able to be shared here, and that permission is given to edit/format the story deemed necessary for inclusion of a module (which may include name alterations for anonymity of characters), and that any formatted stories are ONLY to be shared here on this very thread. Nowhere else.

If you should happen to post a story please include if its: [1st/2nd/3rd person perspective], [Setting], [Primary VoreMethod/ Prominent Fetishes]
=====================================================================
Afterword

This technology is intended to give fledgling writers given a helpful hand in describing their scene. Experienced to have their works enhanced by the AI. Established writers can even make custom module based upon their own stories and to have the AI match their personal writing style. For those with long-running series may include them within a module (do keep these to yourself though), and the AI will begin to learn how your characters think and act in the setting in which they live. Keep in mind it takes hundreds of appearances though multiple books for characters to begin reacting appropriately, a one-off or few chapters is not enough to create this effect.

I want you to understand that this technology is NOT intended to replace quality writers, but to enhance them across the board. Think about artists and how digital technology assists their workflow while enhancing the end result, decade after decade artists have had all the spotlight. This provides a significant boon to writers for an opportunity of comeuppance for having to live in the shadow of artists. At the very least all writers will be able to improve their workflow as this technology develops, summarily increasing their income.

PS - NovelAI is under heavy and active development, making the best use of the AI they have available, once the worldwide computer chip shortage is alleviated they will eventually gain access to more potent AI model, which is will improve the AI's results in every way.
Discord for artists and those who commission them! https://discord.com/invite/F3UDqqAx53
User avatar
AlluringPredation
Participator
 
Posts: 293
Joined: Fri Mar 27, 2015 11:10 am

Re: Unofficial NovelAI General

Postby AlluringPredation » Sun Oct 10, 2021 5:18 pm

Reserved for later compiled contributions.
Discord for artists and those who commission them! https://discord.com/invite/F3UDqqAx53
User avatar
AlluringPredation
Participator
 
Posts: 293
Joined: Fri Mar 27, 2015 11:10 am

Re: Unofficial NovelAI General

Postby Eka » Sun Oct 10, 2021 8:03 pm

This looks really good!

I was going to do some screen cap to help explain but I don't have an account on Novel AI anymore! Thanks for the explanation. This looks in line with everything I saw so far.
User avatar
Eka
Administrator
 
Posts: 4492
Joined: Fri May 13, 2005 10:59 pm
Location: Canada

Re: Unofficial NovelAI General

Postby maleperduis » Tue Oct 12, 2021 1:21 pm

Thanks for your effort on this. I made a very half-hearted attempt at putting together a module a while ago but the results were lacklustre. One question I have is why NovelAI doesn't seem to understand the concept of vore at all, whereas AI Dungeon does. Presumably this is due to a difference in the texts that OpenAI and EleutherAI used to train their GPT models. Do we know how OpenAI picked up vore material, and whether there's a chance to influence the datasets for EleutherAI to pick it up in the same way? E.g. if it's a matter of Eka's being picked up by one web crawler but not another, could some technical change in the site fix that?
User avatar
maleperduis
Intermediate Vorarephile
 
Posts: 358
Joined: Sun Nov 25, 2007 12:00 am
Location: UK

Re: Unofficial NovelAI General

Postby AlluringPredation » Tue Oct 12, 2021 3:23 pm

I'll be trying but without outside support it'll take months to get enough good stories to get a decent module going, and even won't be good without some variety to create a better consensus as to how words are used in vore context. Even with a proper module you're improving results by percentages rather than by leaps and bounds, and module training focuses on the material given it will also get worse at unrelated material. If you're doing it right, this is acceptable, you don't need the AI to focus on space battle and laser-guns when you're wanting swords and sorcery.

NovelAI's uses Eleuther's GPT-Jax 6 billion parameter model (which can run on high-end consumer GPUs), while AID's Dragon utilizes OpenAI's 175 billion parameter model (operating on massive supercomputers), so about 30x the size. NovelAI's model is closer to Open-AI's Curie model (rumored to be 6 billion parameters.). The current chip shortage has put a hold on Eleuther's future AI model training, which is unfortunate as NovelAI is VERY ready to implement larger models. For now they're doing everything they can, including a better fine-tune. NovelAI's fine-tune database is 5GB of professional writing compared to Latitude's quick 30MB scrape from CYOA (including some incredibly offensive material.)

Eleuther's already assembled their Pile of crawled data, everything is ready to process a 22b or 40b AI model, both under the 69b threshold where diminishing returns start kicking in. Even if nothing changes with their data larger AI models can parse data from related topics and begin to form interesting concepts without direct exposure. As for where, random web crawl and reddit. Could we make a change to fix that? Based upon another thread people seem to be generally against that idea, more likely choosing to upload stories as PDF's to completely avoid bots ability to scrape stories for AI use (Hint, hint). The other option is to get some good stories together and convince NovelAI to get some vore in, which is super incredibly unlikely to happen.
Discord for artists and those who commission them! https://discord.com/invite/F3UDqqAx53
User avatar
AlluringPredation
Participator
 
Posts: 293
Joined: Fri Mar 27, 2015 11:10 am

Re: Unofficial NovelAI General

Postby LostCause » Wed Oct 13, 2021 3:39 pm

Real nice thread you put together, glad I spotted it under recent topics. I planned to type up a shorter post but ended up waffling on forever, sorry. Anyway, here's a bunch of random things from my experience with NAI in case anyone finds it useful. In my experience you can get NovelAI to 'understand' vore but it's very hit and miss. One moment it will work fine, then two outputs later loses all coherence. Having lots of vore in the context helps of course, but I find it difficult to know which things to include in lorebook/memory/author's note for best effect. Periods of non-vore make a story much more likely to derail, as a result I've favoured shorter heavily vore-focused scenarios, however this makes it difficult to determine what works best. One thing I can say for sure is that having 'vore' (unbracketed) anywhere within the context only serves to make things worse, producing lots of nonsensical usage of the word. I've also experimented with things such as [ tags: vore; genre: vore ] without any noticeable difference either way. Usually flipping between nucleus sampling and tail free sampling whenever the AI gets 'stuck' has been most effective for me, but this is not ideal at all.

I did a quick scan through AIDs training data back when all the drama was going on and didn't notice any vore content in there, or at the very least not enough for me to find it on a quick scan through. I'm willing to bet 'The Pile' includes vore in some capacity, as both AID and NAI have output unexpectedly good vore text every now and then for me. Personally I expect NAIs currently low capability for vore to be due to the relatively weaker AI model. A more powerful model would be much more effective at 'keeping track' of the scene and producing only relevant text, rather than throwing out loosely related sex outputs and such.

Another thing I've noticed, although it's likely to be a result of my poor writing skills, is the relatively low quality of vore output. I'm willing to bet any vore-related content within 'The Pile' and other general training data is rather basic, without much flourish or flavour. Any time I've pushed the AI to use more flowery and descriptive writing styles (be it with settings, author's note, modules etc) it has felt far more difficult to steer it towards vore and keep it focused. It wants to turn it into sex even more so than usual.

Since the release of biases I've been primarily experimenting with enabling and disabling biases whenever the AI gets stuck on something. For example, biasing against swallowing related words and phrases if the AI keeps outputting that when a character has already been swallowed entirely. This has felt much more effective than positive biasing, but perhaps I'm using biasing incorrectly.

Finally, I'll include a couple excerpts from some old scenarios to prove that, at least for me, NAI is perfectly capable of outputting some decent vore content every now and then. It's just a matter of getting it to that point and then keeping it on track, especially when anything non-vore related happens or has happened previously. This is partly what leads me to believe it to be due to the size of the AI model, as I had this problem much less when I upgraded to dragon from griffin in my AID days. Bear in mind these likely still required a ton of retries, but I made sure to choose excerpts with larger portions of AI outputs. Visual settings are default so white is AI, pink is edits and blue is my input. Made sure to include text that isn't graphic, but it does feature digestion in both. There's a few grammatical errors and weirdness that I either didn't notice or bother to edit at the time, I apologise if it hurts your eyes or brain.

A very brief and to the point swallowing and digestion sequence, with the AI even understanding the double prey situation:[Image link expired so I removed it.]

A much longer scene, where the AI seemed to produce output relevant to both vore and the context of the scene, which I recall being a pleasant surprise at the time. I can't remember if this required significant retries or not, but as you can see by all the pink it required a lot of editing at times. Quick context: I was experimenting to see if the AI would understand a scenario where someone at a party is performing a 'magic trick' in front of the other guests with a willing, but unsuspecting, volunteer. Worked surprisingly well, not sure if I could replicate it however. [Image link expired so I removed it.]
Last edited by LostCause on Tue Sep 26, 2023 12:04 am, edited 1 time in total.
User avatar
LostCause
Participator
 
Posts: 274
Joined: Sat Feb 20, 2021 8:55 am

Re: Unofficial NovelAI General

Postby LostCause » Wed Oct 13, 2021 6:59 pm

My worry with module training is that vore comes in so many variations and people can have vastly different tastes. I'm not sure how beneficial a generic vore module will be, but I have little experience with modules so correct me if I'm wrong. If more specific modules are required then I expect it could be a challenge to collect enough high quality data... I also wonder if the current AI will require separate furry, feral and non-furry/feral vore modules or if all demographics could collaborate successfully to make one module without causing problems.
Last edited by LostCause on Thu Oct 14, 2021 4:23 am, edited 1 time in total.
User avatar
LostCause
Participator
 
Posts: 274
Joined: Sat Feb 20, 2021 8:55 am

Re: Unofficial NovelAI General

Postby AlluringPredation » Wed Oct 13, 2021 9:04 pm

My personal goal is to create a general module. In the AI's current state I'm wondering even how many types it could handle without risk of overlap, and we won't know until we do. I consider everything done here to be preparation for the future, once the 22b or larger AI model becomes available the capability should greatly expand. I'm hoping to write/gather enough stories so people are able to pick and choose among an assortment to create one to suit their needs, because I suspect your concerns are accurate. And help cleaning is appreciated but I feel like a certain amount of activity is going to be needed before others take part.

Having 'vore' anywhere is a terrible idea, one trick is to use...
Detailed description of XXX: XXX
And after a few retries you'll gain an understanding of it's understand, and for vore it's all tied into sexual, most terms you could associate with vore are rather heavily biased towards sexual content. So instead I've biased against sex itself, but it's hard to tell because I'm using my own personal shoddy module that encourages vore, even if rather haphazardly.

Some of the tricks I utilized Javascript for in AID I'm still waiting to make an appearance in NAI, they're on the list but unknown/low priority.
I'm also keen on experimenting creating a tag system for vore within a specific module, if it's even possible and effective, would require some careful writing constrained to exactly 256 token chunks at a time. Basically ~175 word mini-stories. I've been hoping they'd come out with the method to continue training modules to make this considerably easier, and provide a way to enhance current modules with enough vore training to make writing a story considerably easier.
Discord for artists and those who commission them! https://discord.com/invite/F3UDqqAx53
User avatar
AlluringPredation
Participator
 
Posts: 293
Joined: Fri Mar 27, 2015 11:10 am

Re: Unofficial NovelAI General

Postby JadeTheDeer » Wed Oct 13, 2021 10:59 pm

Hi, for those interested in toying around with a non-paid AI, KoboldAI exists:

https://github.com/KoboldAI/KoboldAI-Client

It takes a bit to get it set up using google's cloud AI thing but you can do it for free and play around with it. It works pretty well, but takes some toying around.
User avatar
JadeTheDeer
Somewhat familiar
 
Posts: 99
Joined: Wed Oct 31, 2012 4:57 pm

Re: Unofficial NovelAI General

Postby LostCause » Thu Oct 14, 2021 4:29 am

AlluringPredation wrote:Having 'vore' anywhere is a terrible idea, one trick is to use...
Detailed description of XXX: XXX
And after a few retries you'll gain an understanding of it's understand, and for vore it's all tied into sexual, most terms you could associate with vore are rather heavily biased towards sexual content. So instead I've biased against sex itself, but it's hard to tell because I'm using my own personal shoddy module that encourages vore, even if rather haphazardly.

I'm not quite clear on what you mean here. Can you share an example? I could use your example myself to see if it works without any module set too.

I wonder how effective it might be to copy the tag formatting supposedly used in the finetune. Different vore types, lethal, non-lethal etc could be tagged with square brackets before each bit of data, allowing users to copy said tags into their memory or author's note. Could extend this to themes and genres too if there's eventually enough data to support that. Perhaps this could aid in keeping the AI on track when using a general vore module? I wasn't sure if this is what you meant when you mentioned a tag system, as I'm not familiar with why it would need to be exactly 256 tokens.

and provide a way to enhance current modules with enough vore training to make writing a story considerably easier.

That'd be great, but I feel it's unlikely due to concerns with plagiarism, piracy and such I've seen when discussing Sigurd's main finetune data. I expect this concern would extend to custom modules also. Maybe instead you'd have to convince the creator to share their dataset if there's a module in particular you like, then re-train it with some vore added in.
User avatar
LostCause
Participator
 
Posts: 274
Joined: Sat Feb 20, 2021 8:55 am

Re: Unofficial NovelAI General

Postby AlluringPredation » Thu Oct 14, 2021 8:43 am

foxygrandpa wrote:Hi, for those interested in toying around with a non-paid AI, KoboldAI exists

It does, if you happen to have a powerful enough graphics card, or the cash to spend while prices are 3x what they should be. I just want a new rig so I can set up a devoted art station but don't want to get ripped off.

LostCause wrote:I'm not quite clear on what you mean here. Can you share an example? I could use your example myself to see if it works without any module set too.

Sure, asking the AI about stuff is a powerful tool. Copy/pasted a few examples, _vore and _Vore are different tokens, and as you can see they are not what we're looking for. I have a personal module that often gets the idea right, but is spastic and gets confused about whose doing what as it was trained heavily on some poor quality content (my old AID stories).
Image

Training data fed to modules is broken up into 256 chunks of tokens and shuffled all around. You wouldn't absolutely need to be this careful, but I want to experiment, anticipate how information will be broken up means I can keep the tags at the top of each chunk and control the exact text associated with that tag. I'm leery of doing too much tagging as it may lose focus of become confused, mainly just want to get oral vore working first before expanding further, and lorebook can be the best way still. As you can get vore to work with lorebook alone.

Natural Language AI models have varying levels of Few-Shot, One-Shot, and Zero-Shot capability, properly trained/focused AI models need fewer.
Before Latitude stupified Dragon, it possessed a strong One-Shot and some Zero-Shot potential. One-shot means it only needed one example, and Zero-shot (though very low) means it could pull information from related sources. You could do some crazy stuff, like to get lamia working the top line of this next image was the note I gave to the AI and below is it's interpretation.
Image

Full 175b GPT-3 is WAY overpowered for our use case given cost of current tech, and we likely don't need something that can take pseudocode and implement it flawlessly for story guidance. GPT-Jax is generally a few shot model because lower parameter count, it needs a few examples to prime it for the desired situation, so it's a requirement to use multiple lorebook entries. Typically I been writing to encourage the correct output, when the AI fails I continually modify the lorebook until it can at least get it correct sometimes, but it's tedious and lacking free time to give polish.

My method in AID's Lorebook equivalent was stacking vore actions in order, always encouraging the next step where keywords were concerned. I often just keep the initiation of vore in constant context, the beginning of ingestion. When keywords crop up that signal vore has begun I load up the swallowing portion, and when that occurs I have it load up the last part. Digestion, Endo, whatever your flavor.
Discord for artists and those who commission them! https://discord.com/invite/F3UDqqAx53
User avatar
AlluringPredation
Participator
 
Posts: 293
Joined: Fri Mar 27, 2015 11:10 am

Re: Unofficial NovelAI General

Postby LostCause » Fri Nov 19, 2021 11:40 am

How've people been finding Sigurd v4? I had a great experience on the first day, managed to get noticeably more vivid descriptions and better writing quality in general... although I haven't been able to replicate this in the past couple days. Need to take the time to try to figure out what appeared to work well I suppose. Messed with making a quick module too, but I have nowhere near enough training steps or data for a module to make any noticeable difference. Feel that it'd need an excessively large amount of focused, high quality data to "teach" the current AI vore through a module without negatively impacting writing quality or variety. In my opinion it's not worth pursuing that route until a more powerful AI is released further down the line. Even then I reckon it'd have to be a larger collaborative effort for something as niche/unusual as vore.

Most recently I've been messing with negative biases to prevent the all too common switches to ripping and tearing, sex, vampirism, kissing, spitting out prey etc. Unfortunately this doesn't seem to noticeably improve NAI's consistency in maintaining/starting vore scenes, but it's something. Besides, it's funny to see all the roundabouts ways the AI uses to get around the biases. I'd never seen "mons veneris" used to refer to a vagina until today lol. Anyone else experimented with biases and found anything useful? Perhaps a list of positive biases that have worked well for them? That's something I haven't played around with yet because I feel I'd cause stories to become more repetitive.
User avatar
LostCause
Participator
 
Posts: 274
Joined: Sat Feb 20, 2021 8:55 am

Re: Unofficial NovelAI General

Postby Redley » Sat Nov 20, 2021 11:31 pm

My personal experience with v4 was pretty enjoyable. Maybe I just got lucky, but the vore narrative became more consistent. Of course the AI is still confused about the stages and doesn't completely understand the concept of being digested. But now it seems to be much better at arguing and discussing about vore, like in scenes where the predator is trying to talk their prey into accepting their role of food. Now NAI looks as good at it as AID was.
Also after the release I've done some tests of modules. I have several vore modules trined from my personal favorites backlog, like vore with sentient predator, vore with feral creature like snake, worm, etc. I've trined them all over on v4 and tested some scenarios with empty memory, lore book and author's note. In my experience the story is now much easier falling into vore route. With v3 I had to force vore theme to come up, but now it seems to be working on it's own.
The main problem I have with both NAI and AID is their bad understanding of vore process. They both are much better at taling about vore than describing vore. Maybe I'll do some research to find a way to create strict rules that AI will follow, but I doubt it's possible at the moment.
User avatar
Redley
Been posting for a bit
 
Posts: 56
Joined: Mon Jun 08, 2009 9:09 am

Re: Unofficial NovelAI General

Postby LostCause » Sun Nov 21, 2021 5:29 pm

Interesting. It's still been off and on for me. Sometimes I can consistently keep a vore scene going, while other times it fails to keep up for even one output and I'm writing almost the entire thing myself. I've even had times in the past (before v4 too) where it described graphic digestion well, but I can never consistently replicate it. Personally I mostly avoid dialogue as it rarely makes sense and often degrades output quality simply by existing in the context. I'm surprised someone has had the opposite experience with dialogue.

An irritating issue I've been having is that it loves to switch to the predator forcing themselves down the other's throat instead of the other way around... I've tried reinforcing the 'roles' in the past but using words such as predator usually makes it turn into biting heads off, tearing out chunks of flesh etc haha. Why does the AI seemingly have more training data where one forces themselves down another's throat than the other way around...? Very odd to me. Also I've noticed the AI almost always wants to end swallowing scenes with the head being swallowed, I guess a large portion of whatever is in the training data is feet-first? The head can be in the stomach already and it'll still keep outputting things in feet-first order. It's strange. Could just be me though.
User avatar
LostCause
Participator
 
Posts: 274
Joined: Sat Feb 20, 2021 8:55 am

Re: Unofficial NovelAI General

Postby AlluringPredation » Mon Nov 22, 2021 11:46 am

Been a bit since I checked out this thread.

v4 has been a modest boost to the capabilities to the AI, not magical but noticeable improvement. Dialogue though has seen the greatest improvement, as was their focus. When testing the general description of vore the base AI was able to get it right more than it used to, though many elements still remain incorrect. Usually, I have to keep randomness above 0.70 to get decent vore results. 1.00 is where they get really good if you can wrangle the AI from becoming nonsensical and dealing with retyping output.

Users have noticed an uptick of Eleuther activity and may be working on a 20B model titled GPT-Thicc (https://wandb.ai/eleutherai/gpt-thicc), Yes...THICC. If true this would provide a noticeable improvement over the current GPT-J 6B. Being in its own weight class of AI models (Other groups jump directly from ~6B to 175B) it's exciting to me what it may be capable of since we have no comparison. But don't expect anything until mid-January, at the VERY earliest.

With modules I've had some good initial success with taking advantage of how AI module training shuffles data into 256 token chunks, writing up short descriptions of vore with a heading of [Tags: vore ] while appending it to the beginning of my module training data while making sure each Tagged chunk is exactly 256 tokens long (including newlines). This could greatly reduce the number of examples needed by having the AI hyper-focus on specific parts of the process taking cues from Author's notes [Tags: ]. Also combined this with lorebook's phrasebias function and regex I have NAI reinforce word usage for different tagged stops. Plotting and planning this out took an entire evening for just the first iteration. Still some way off from being ready as the stomach portion doesn't work well at all. Quick test I did:
Attachments
NAIv4Example.png
Discord for artists and those who commission them! https://discord.com/invite/F3UDqqAx53
User avatar
AlluringPredation
Participator
 
Posts: 293
Joined: Fri Mar 27, 2015 11:10 am

Re: Unofficial NovelAI General

Postby LostCause » Mon Nov 22, 2021 3:43 pm

I wonder what's going on with that gpt-thicc, I've seen no discussion of it in the Eleuther discord, aside from someone asking if it's real and getting mostly ignored. I'm guessing it's not GPT-J if it's using GPUs instead of TPUs, in which case I doubt this is a sign of a large model being trained soon. As far as I heard they've been ready to begin training a larger model for a while now, but can't acquire the GPUs. So, since I don't want to get my hopes up too soon, I'm guessing this is something along the lines of testing new changes and optimisations, rather than gearing up for full scale model training in the near future... but I've only scratched the surface level of this stuff so take what I say with a grain of salt. EDIT: Although, there is a 20B pre-training report on that wandb from 4 days ago so who knows.

Neat experimentation there. Really seems to help keep the AI on track more effectively. Shame about the usual AI weirdness like there being two tongues for a moment though. I'm curious what sort of settings you used there, and whether you've managed to replicate this much? Have you tested other perspectives? I'm guessing you'd want to have these chunks for each perspective and possibly tense too for maximum compatibility?

As a side note, have you found using vore as the keyword to be effective without any detrimental side-effects? I sometimes found the AI does strange things when it finds "vore" within the context, even as a tag. Some examples being bringing in "Vores" as a race or the name of an activity.
User avatar
LostCause
Participator
 
Posts: 274
Joined: Sat Feb 20, 2021 8:55 am

Re: Unofficial NovelAI General

Postby AlluringPredation » Mon Nov 22, 2021 4:39 pm

Eleuther doesn't like talking about things and getting people's hopes up and creating expectations. A lot of people on the 4chan/vg threads and the NAI discord have been talking about it. They state everyone figures it out anyways, so they just say nothing and let people make their own assumptions. I'm erring for optimism because I need the copium.

Can I replicate this? Absolutely! To the point I can't get it to stop eating and swallowing the same character and proceed being in a stomach. So that's the big hurdle.

As for that screenshot; I think it was going for a forked tongue as it associated dragons with reptiles, just couldn't explain it properly, so I saw that as an interesting positive. It's only really working for 2nd person as that can hit either gender prey a lot easier, and what's mostly what's in the module training data. I might be able to use regex to scan for gender keywords and apply proper lorebooks for 3rd person him/her but that's down the road.

Vore is typically VERY detrimental with your experience being common, ` vore` (with a preceding whitespace) provided the best base result (but still bad). The specific way I've started tagging my module training data is the defining difference in getting the term to work, I've also set up NAI to boost the bias of associated words when [ Tags: vore ] appears in the author's note (using cascading activation) with regex. Witness the insanity!
Attachments
NAIv4Example.png
Discord for artists and those who commission them! https://discord.com/invite/F3UDqqAx53
User avatar
AlluringPredation
Participator
 
Posts: 293
Joined: Fri Mar 27, 2015 11:10 am

Re: Unofficial NovelAI General

Postby LostCause » Tue Nov 23, 2021 1:26 pm

Fair point on Eleuther. It's just as far as I was aware a fair chuck of their communication about their projects was done in their public discord, so I figured there'd be at least a tiny hint if they were gearing up to train a model, but what you're saying there seems more likely to be honest.

As for the rest of your post... Good lord, that's an insane amount of work. Best of luck with that, I hope it all pays off for you and doesn't get torn apart too much if/when a stronger model comes around. Have you created negative biases for the initial tokens of positively biased words such as Shoved? I'm curious why you've had to do this, if you don't mind sharing. Also would it not be beneficial to check 'ensure completion after start' since many of those are multi-token words?

The Pocket Notes lorebook has a tool for converting text to other perspectives, perhaps that could be helpful in some way? I've not tried that tool personally, nor have I delved into Pocket Notes much, but I've heard great things about it.

About the tongue, I was thinking it'd be something along the lines of a forked tongue but that's a pretty poor way to describe it. Chances are the AI would confuse itself once it puts that two-tongue line in the context lol.

Good luck with all your work!
User avatar
LostCause
Participator
 
Posts: 274
Joined: Sat Feb 20, 2021 8:55 am

Re: Unofficial NovelAI General

Postby AlluringPredation » Tue Nov 23, 2021 8:23 pm

Most of the work was pre-planning utilization of the bias. I took all the vore associated words I could think of and their use cases, organizing them all on a spreadsheet while I checked each word's token use. I split them all into specific tags like ingest, swallow, and digestion; with absorption added later. I also included negative bias to eating/swallowing for the digestion part to force progression. Because of this, I can change one word in the author's note to completely switch gears. In that planning I have been applying negative balance to multi-token words used in multiple. I stay away from 'ensure completion' because it will eliminate words that begin with the same token; including names and made-up terminology. Instead I boost the follow-up token by itself, or could apply the word bias twice and negative the initial token. Lorebook context helps fill in gaps while providing examples for better results.

The only time I use 'ensure completion' is when there's a phrase/line I liked in output but plan to retry. So I place that phrase in my bias and see how it's utilized, sometimes it's wonky, and sometimes it creates amazing results. I also have done this to dialogue/phrases I've personally thought up, letting the AI apply it where it will. The dialogue especially has improved my results.

I'm of the mindset that any work done now will be exponentially improved with a new AI model. As for why, as above, I can now change a single word shift the AI's focus to what I want. Eventually I plan to share my bias/lorebook setup, and module later, so others can give it a whirl. Most everything I'm doing in based on the default settings to make it plug-and-play as possible.
Discord for artists and those who commission them! https://discord.com/invite/F3UDqqAx53
User avatar
AlluringPredation
Participator
 
Posts: 293
Joined: Fri Mar 27, 2015 11:10 am

Re: Unofficial NovelAI General

Postby AlluringPredation » Tue Nov 23, 2021 10:22 pm

I've also discovered that bias phrases with the same initial token are used sequentially. So there is no need or purpose to provide negative balance. And also presents some issues trying to bias tokens that have overlapping initial tokens.
Discord for artists and those who commission them! https://discord.com/invite/F3UDqqAx53
User avatar
AlluringPredation
Participator
 
Posts: 293
Joined: Fri Mar 27, 2015 11:10 am

Next

Return to General Vore Discussion

Who is online

Users browsing this forum: Canzo, JackSpades